You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/06/21 13:52:02 UTC
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1347 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1347/
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:56460/md/q/collection1lastClient and got 257 from http://127.0.0.1:56467/md/q/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:56460/md/q/collection1lastClient and got 257 from http://127.0.0.1:56467/md/q/collection1
at __randomizedtesting.SeedInfo.seed([4FBBD6639EBFF2F4:CE5D587BE9E092C8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9659 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 740960 T1637 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /md/q
[junit4:junit4] 2> 740967 T1637 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solrtest-SyncSliceTest-1371814364376
[junit4:junit4] 2> 740968 T1637 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 740969 T1638 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 741070 T1637 oasc.ZkTestServer.run start zk server on port:56441
[junit4:junit4] 2> 741071 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 741075 T1644 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c65d8e3 name:ZooKeeperConnection Watcher:127.0.0.1:56441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 741075 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 741076 T1637 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 741088 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 741091 T1646 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7848e1e0 name:ZooKeeperConnection Watcher:127.0.0.1:56441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 741091 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 741092 T1637 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 741101 T1637 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 741104 T1637 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 741107 T1637 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 741110 T1637 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 741111 T1637 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 741123 T1637 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 741123 T1637 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 741228 T1637 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 741229 T1637 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 741232 T1637 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 741233 T1637 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 741242 T1637 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 741243 T1637 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 741246 T1637 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 741246 T1637 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 741249 T1637 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 741250 T1637 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 741253 T1637 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 741253 T1637 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 741261 T1637 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 741262 T1637 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 741527 T1637 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 741532 T1637 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56449
[junit4:junit4] 2> 741533 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 741534 T1637 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 741534 T1637 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371814364674
[junit4:junit4] 2> 741535 T1637 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371814364674/solr.xml
[junit4:junit4] 2> 741535 T1637 oasc.CoreContainer.<init> New CoreContainer 1674828849
[junit4:junit4] 2> 741536 T1637 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371814364674/'
[junit4:junit4] 2> 741536 T1637 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371814364674/'
[junit4:junit4] 2> 741670 T1637 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 741671 T1637 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 741671 T1637 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 741672 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 741672 T1637 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 741673 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 741673 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 741674 T1637 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 741674 T1637 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 741675 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 741694 T1637 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 741695 T1637 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56441/solr
[junit4:junit4] 2> 741695 T1637 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 741696 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 741710 T1657 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@442c02d0 name:ZooKeeperConnection Watcher:127.0.0.1:56441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 741710 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 741713 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 741727 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 741729 T1659 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43d20d69 name:ZooKeeperConnection Watcher:127.0.0.1:56441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 741729 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 741732 T1637 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 741773 T1637 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 741816 T1637 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 741857 T1637 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56449_md%2Fq
[junit4:junit4] 2> 741883 T1637 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56449_md%2Fq
[junit4:junit4] 2> 741886 T1637 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 741968 T1637 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 742021 T1637 oasc.Overseer.start Overseer (id=89903226184204291-127.0.0.1:56449_md%2Fq-n_0000000000) starting
[junit4:junit4] 2> 742086 T1637 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 742146 T1661 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 742147 T1637 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 742217 T1637 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 742219 T1637 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 742222 T1660 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 742226 T1662 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 742227 T1662 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 743725 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 743726 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56449_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56449/md/q"}
[junit4:junit4] 2> 743726 T1660 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 743727 T1660 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 743746 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 744230 T1662 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371814364674/collection1
[junit4:junit4] 2> 744230 T1662 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 744231 T1662 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 744231 T1662 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 744233 T1662 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371814364674/collection1/'
[junit4:junit4] 2> 744234 T1662 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1371814364674/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 744235 T1662 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1371814364674/collection1/lib/README' to classloader
[junit4:junit4] 2> 744289 T1662 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 744351 T1662 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 744453 T1662 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 744460 T1662 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 745125 T1662 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 745126 T1662 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 745127 T1662 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 745138 T1662 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 745142 T1662 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 745155 T1662 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 745160 T1662 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 745164 T1662 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 745164 T1662 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 745165 T1662 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 745165 T1662 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 745166 T1662 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 745166 T1662 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 745166 T1662 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 745210 T1662 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371814364674/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/
[junit4:junit4] 2> 745210 T1662 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7692fed2
[junit4:junit4] 2> 745211 T1662 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 745212 T1662 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data
[junit4:junit4] 2> 745213 T1662 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index/
[junit4:junit4] 2> 745214 T1662 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 745215 T1662 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index
[junit4:junit4] 2> 745223 T1662 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ad338e8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 745223 T1662 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 745226 T1662 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 745227 T1662 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 745227 T1662 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 745228 T1662 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 745228 T1662 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 745229 T1662 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 745229 T1662 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 745229 T1662 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 745230 T1662 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 745245 T1662 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 745254 T1662 oass.SolrIndexSearcher.<init> Opening Searcher@60b5ce43 main
[junit4:junit4] 2> 745255 T1662 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 745255 T1662 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 745259 T1663 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60b5ce43 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 745261 T1662 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 745261 T1662 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56449/md/q collection:control_collection shard:shard1
[junit4:junit4] 2> 745262 T1662 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 745278 T1662 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 745287 T1662 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 745287 T1662 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 745287 T1662 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56449/md/q/collection1/
[junit4:junit4] 2> 745288 T1662 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 745288 T1662 oasc.SyncStrategy.syncToMe http://127.0.0.1:56449/md/q/collection1/ has no replicas
[junit4:junit4] 2> 745288 T1662 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56449/md/q/collection1/
[junit4:junit4] 2> 745288 T1662 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 746752 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 746842 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 746936 T1662 oasc.ZkController.register We are http://127.0.0.1:56449/md/q/collection1/ and leader is http://127.0.0.1:56449/md/q/collection1/
[junit4:junit4] 2> 746936 T1662 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56449/md/q
[junit4:junit4] 2> 746936 T1662 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 746936 T1662 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 746937 T1662 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 746939 T1662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 746940 T1637 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 746940 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 746941 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 746958 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 746959 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 746962 T1666 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28a1638f name:ZooKeeperConnection Watcher:127.0.0.1:56441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 746962 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 746963 T1637 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 746965 T1637 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 747233 T1637 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 747236 T1637 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56460
[junit4:junit4] 2> 747237 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 747238 T1637 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 747238 T1637 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371814370374
[junit4:junit4] 2> 747239 T1637 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371814370374/solr.xml
[junit4:junit4] 2> 747239 T1637 oasc.CoreContainer.<init> New CoreContainer 2107635280
[junit4:junit4] 2> 747240 T1637 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371814370374/'
[junit4:junit4] 2> 747240 T1637 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371814370374/'
[junit4:junit4] 2> 747367 T1637 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 747368 T1637 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 747369 T1637 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 747369 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 747370 T1637 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 747370 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 747371 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 747371 T1637 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 747372 T1637 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 747372 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 747389 T1637 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 747390 T1637 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56441/solr
[junit4:junit4] 2> 747390 T1637 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 747391 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 747397 T1677 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b400cfe name:ZooKeeperConnection Watcher:127.0.0.1:56441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 747397 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 747407 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 747419 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 747421 T1679 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@75e30834 name:ZooKeeperConnection Watcher:127.0.0.1:56441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 747422 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 747433 T1637 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 748420 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 748421 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56449_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56449/md/q"}
[junit4:junit4] 2> 748437 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 748437 T1637 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56460_md%2Fq
[junit4:junit4] 2> 748437 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 748437 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 748439 T1637 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56460_md%2Fq
[junit4:junit4] 2> 748449 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 748450 T1679 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 748450 T1666 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 748450 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 748450 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 748452 T1659 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 748457 T1680 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 748458 T1680 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 749941 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 749942 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56460_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56460/md/q"}
[junit4:junit4] 2> 749942 T1660 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 749942 T1660 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 749956 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 749956 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 749956 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 750461 T1680 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371814370374/collection1
[junit4:junit4] 2> 750461 T1680 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 750462 T1680 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 750462 T1680 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 750463 T1680 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371814370374/collection1/'
[junit4:junit4] 2> 750465 T1680 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1371814370374/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 750465 T1680 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1371814370374/collection1/lib/README' to classloader
[junit4:junit4] 2> 750517 T1680 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 750577 T1680 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 750679 T1680 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 750685 T1680 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 751318 T1680 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 751319 T1680 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 751320 T1680 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 751332 T1680 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 751336 T1680 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 751350 T1680 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 751355 T1680 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 751359 T1680 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 751360 T1680 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 751360 T1680 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 751360 T1680 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 751361 T1680 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 751362 T1680 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 751362 T1680 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 751362 T1680 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371814370374/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/
[junit4:junit4] 2> 751363 T1680 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7692fed2
[junit4:junit4] 2> 751363 T1680 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 751364 T1680 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1
[junit4:junit4] 2> 751365 T1680 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index/
[junit4:junit4] 2> 751366 T1680 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 751367 T1680 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index
[junit4:junit4] 2> 751374 T1680 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 751374 T1680 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 751378 T1680 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 751378 T1680 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 751379 T1680 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 751380 T1680 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 751381 T1680 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 751381 T1680 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 751381 T1680 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 751382 T1680 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 751382 T1680 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 751399 T1680 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 751408 T1680 oass.SolrIndexSearcher.<init> Opening Searcher@163f2da main
[junit4:junit4] 2> 751409 T1680 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 751410 T1680 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 751415 T1681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@163f2da main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 751417 T1680 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 751418 T1680 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56460/md/q collection:collection1 shard:shard1
[junit4:junit4] 2> 751418 T1680 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 751434 T1680 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 751446 T1680 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 751446 T1680 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 751446 T1680 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 751447 T1680 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 751447 T1680 oasc.SyncStrategy.syncToMe http://127.0.0.1:56460/md/q/collection1/ has no replicas
[junit4:junit4] 2> 751447 T1680 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 751447 T1680 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 751460 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 751482 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 751482 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 751482 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 751510 T1680 oasc.ZkController.register We are http://127.0.0.1:56460/md/q/collection1/ and leader is http://127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 751510 T1680 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56460/md/q
[junit4:junit4] 2> 751510 T1680 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 751510 T1680 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 751511 T1680 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 751512 T1680 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 751513 T1637 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 751514 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 751514 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 751801 T1637 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 751804 T1637 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56463
[junit4:junit4] 2> 751805 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 751805 T1637 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 751806 T1637 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371814374941
[junit4:junit4] 2> 751806 T1637 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371814374941/solr.xml
[junit4:junit4] 2> 751807 T1637 oasc.CoreContainer.<init> New CoreContainer 839456568
[junit4:junit4] 2> 751808 T1637 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371814374941/'
[junit4:junit4] 2> 751808 T1637 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371814374941/'
[junit4:junit4] 2> 751936 T1637 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 751937 T1637 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 751937 T1637 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 751938 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 751938 T1637 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 751939 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 751939 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 751940 T1637 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 751940 T1637 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 751941 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 751959 T1637 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 751960 T1637 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56441/solr
[junit4:junit4] 2> 751961 T1637 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 751962 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 752028 T1693 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d96e7ba name:ZooKeeperConnection Watcher:127.0.0.1:56441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 752029 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 752063 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 752077 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 752079 T1695 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ced7d94 name:ZooKeeperConnection Watcher:127.0.0.1:56441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 752079 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 752142 T1637 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 752987 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 752988 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56460_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56460/md/q"}
[junit4:junit4] 2> 753006 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 753006 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 753006 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 753006 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 753145 T1637 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56463_md%2Fq
[junit4:junit4] 2> 753148 T1637 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56463_md%2Fq
[junit4:junit4] 2> 753150 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 753150 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 753151 T1679 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 753151 T1666 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 753151 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 753151 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 753152 T1695 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 753158 T1659 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 753164 T1696 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 753164 T1696 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 754510 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 754511 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56463_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56463/md/q"}
[junit4:junit4] 2> 754511 T1660 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 754511 T1660 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 754530 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 754530 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 754530 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 754530 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 755167 T1696 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371814374941/collection1
[junit4:junit4] 2> 755167 T1696 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 755168 T1696 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 755168 T1696 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 755169 T1696 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371814374941/collection1/'
[junit4:junit4] 2> 755171 T1696 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1371814374941/collection1/lib/README' to classloader
[junit4:junit4] 2> 755172 T1696 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1371814374941/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 755224 T1696 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 755284 T1696 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 755385 T1696 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 755391 T1696 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 755992 T1696 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 755993 T1696 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 755994 T1696 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 756002 T1696 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 756004 T1696 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 756016 T1696 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 756019 T1696 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 756022 T1696 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 756023 T1696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 756023 T1696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 756023 T1696 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 756024 T1696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 756024 T1696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 756025 T1696 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 756025 T1696 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371814374941/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/
[junit4:junit4] 2> 756025 T1696 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7692fed2
[junit4:junit4] 2> 756026 T1696 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 756026 T1696 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2
[junit4:junit4] 2> 756027 T1696 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index/
[junit4:junit4] 2> 756027 T1696 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 756028 T1696 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index
[junit4:junit4] 2> 756033 T1696 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d4d9bc3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 756033 T1696 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 756036 T1696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 756036 T1696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 756037 T1696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 756037 T1696 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 756038 T1696 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 756038 T1696 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 756038 T1696 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 756039 T1696 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 756039 T1696 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 756049 T1696 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 756056 T1696 oass.SolrIndexSearcher.<init> Opening Searcher@1430ff39 main
[junit4:junit4] 2> 756057 T1696 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 756057 T1696 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 756060 T1697 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1430ff39 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 756062 T1696 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 756063 T1696 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56463/md/q collection:collection1 shard:shard1
[junit4:junit4] 2> 756065 T1696 oasc.ZkController.register We are http://127.0.0.1:56463/md/q/collection1/ and leader is http://127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 756065 T1696 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56463/md/q
[junit4:junit4] 2> 756066 T1696 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 756066 T1696 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C450 name=collection1 org.apache.solr.core.SolrCore@4bde63d4 url=http://127.0.0.1:56463/md/q/collection1 node=127.0.0.1:56463_md%2Fq C450_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:56463_md%2Fq, base_url=http://127.0.0.1:56463/md/q}
[junit4:junit4] 2> 756066 T1698 C450 P56463 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 756066 T1696 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 756067 T1698 C450 P56463 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 756067 T1698 C450 P56463 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 756067 T1698 C450 P56463 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 756068 T1637 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 756068 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 756068 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 756069 T1698 C450 P56463 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 756086 T1669 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 756260 T1637 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 756262 T1637 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56467
[junit4:junit4] 2> 756263 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 756263 T1637 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 756264 T1637 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371814379491
[junit4:junit4] 2> 756264 T1637 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371814379491/solr.xml
[junit4:junit4] 2> 756264 T1637 oasc.CoreContainer.<init> New CoreContainer 184639518
[junit4:junit4] 2> 756265 T1637 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371814379491/'
[junit4:junit4] 2> 756265 T1637 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371814379491/'
[junit4:junit4] 2> 756349 T1637 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 756350 T1637 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 756350 T1637 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 756351 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 756351 T1637 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 756351 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 756352 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 756352 T1637 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 756352 T1637 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 756353 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 756364 T1637 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 756364 T1637 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56441/solr
[junit4:junit4] 2> 756364 T1637 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 756365 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 756368 T1710 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@75448e27 name:ZooKeeperConnection Watcher:127.0.0.1:56441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 756369 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 756384 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 756395 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 756397 T1712 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@306e8128 name:ZooKeeperConnection Watcher:127.0.0.1:56441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 756397 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 756406 T1637 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 757409 T1637 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56467_md%2Fq
[junit4:junit4] 2> 757415 T1637 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56467_md%2Fq
[junit4:junit4] 2> 757418 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 757418 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 757418 T1666 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 757418 T1679 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 757418 T1712 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 757419 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 757419 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 757433 T1659 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 757433 T1695 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 757438 T1713 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 757439 T1713 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 757536 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 757537 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56463_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56463/md/q"}
[junit4:junit4] 2> 757541 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56467_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56467/md/q"}
[junit4:junit4] 2> 757541 T1660 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 757541 T1660 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 757560 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 757560 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 757560 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 757560 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 757560 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 758088 T1669 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 758088 T1669 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=3&state=recovering&nodeName=127.0.0.1:56463_md%252Fq&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 758441 T1713 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371814379491/collection1
[junit4:junit4] 2> 758441 T1713 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 758442 T1713 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 758442 T1713 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 758443 T1713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371814379491/collection1/'
[junit4:junit4] 2> 758445 T1713 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1371814379491/collection1/lib/README' to classloader
[junit4:junit4] 2> 758446 T1713 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1371814379491/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 758497 T1713 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 758558 T1713 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 758659 T1713 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 758665 T1713 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 759298 T1713 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 759299 T1713 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 759300 T1713 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 759312 T1713 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 759317 T1713 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 759330 T1713 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 759335 T1713 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 759339 T1713 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 759340 T1713 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 759340 T1713 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 759341 T1713 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 759341 T1713 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 759342 T1713 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 759342 T1713 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 759342 T1713 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371814379491/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/
[junit4:junit4] 2> 759343 T1713 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7692fed2
[junit4:junit4] 2> 759343 T1713 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 759345 T1713 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3
[junit4:junit4] 2> 759345 T1713 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index/
[junit4:junit4] 2> 759346 T1713 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 759347 T1713 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index
[junit4:junit4] 2> 759381 T1713 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff4edc0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 759381 T1713 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 759385 T1713 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 759385 T1713 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 759386 T1713 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 759387 T1713 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 759387 T1713 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 759388 T1713 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 759388 T1713 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 759389 T1713 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 759389 T1713 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 759406 T1713 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 759415 T1713 oass.SolrIndexSearcher.<init> Opening Searcher@7732e562 main
[junit4:junit4] 2> 759417 T1713 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 759417 T1713 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 759422 T1714 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7732e562 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 759425 T1713 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 759425 T1713 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56467/md/q collection:collection1 shard:shard1
[junit4:junit4] 2> 759428 T1713 oasc.ZkController.register We are http://127.0.0.1:56467/md/q/collection1/ and leader is http://127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 759428 T1713 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56467/md/q
[junit4:junit4] 2> 759429 T1713 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 759429 T1713 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C451 name=collection1 org.apache.solr.core.SolrCore@1ca0bd7d url=http://127.0.0.1:56467/md/q/collection1 node=127.0.0.1:56467_md%2Fq C451_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:56467_md%2Fq, base_url=http://127.0.0.1:56467/md/q}
[junit4:junit4] 2> 759429 T1715 C451 P56467 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 759430 T1713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 759430 T1715 C451 P56467 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 759430 T1715 C451 P56467 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 759431 T1715 C451 P56467 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 759431 T1637 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 759432 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 759432 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 759437 T1715 C451 P56467 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 759454 T1670 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 759763 T1637 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 759766 T1637 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56477
[junit4:junit4] 2> 759767 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 759768 T1637 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 759768 T1637 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371814382860
[junit4:junit4] 2> 759769 T1637 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371814382860/solr.xml
[junit4:junit4] 2> 759769 T1637 oasc.CoreContainer.<init> New CoreContainer 993616770
[junit4:junit4] 2> 759770 T1637 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371814382860/'
[junit4:junit4] 2> 759770 T1637 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371814382860/'
[junit4:junit4] 2> 759898 T1637 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 759899 T1637 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 759899 T1637 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 759900 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 759900 T1637 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 759901 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 759901 T1637 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 759902 T1637 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 759903 T1637 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 759903 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 759920 T1637 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 759921 T1637 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56441/solr
[junit4:junit4] 2> 759921 T1637 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 759923 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 759925 T1727 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a713d22 name:ZooKeeperConnection Watcher:127.0.0.1:56441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 759926 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 759928 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 759940 T1637 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 759942 T1729 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f00f9cc name:ZooKeeperConnection Watcher:127.0.0.1:56441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 759942 T1637 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 759954 T1637 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C452 name=collection1 org.apache.solr.core.SolrCore@4bde63d4 url=http://127.0.0.1:56463/md/q/collection1 node=127.0.0.1:56463_md%2Fq C452_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:56463_md%2Fq, base_url=http://127.0.0.1:56463/md/q}
[junit4:junit4] 2> 760090 T1698 C452 P56463 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:56460/md/q/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 760090 T1698 C452 P56463 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56463/md/q START replicas=[http://127.0.0.1:56460/md/q/collection1/] nUpdates=100
[junit4:junit4] 2> 760090 T1698 C452 P56463 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 760091 T1698 C452 P56463 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 760091 T1698 C452 P56463 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 760091 T1698 C452 P56463 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 760092 T1698 C452 P56463 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 760092 T1698 C452 P56463 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:56460/md/q/collection1/. core=collection1
[junit4:junit4] 2> 760092 T1698 C452 P56463 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C453 name=collection1 org.apache.solr.core.SolrCore@4b21f7dc url=http://127.0.0.1:56460/md/q/collection1 node=127.0.0.1:56460_md%2Fq C453_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56460_md%2Fq, base_url=http://127.0.0.1:56460/md/q, leader=true}
[junit4:junit4] 2> 760099 T1671 C453 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 760106 T1672 C453 P56460 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760111 T1672 C453 P56460 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 760111 T1672 C453 P56460 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 760116 T1672 C453 P56460 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 760116 T1672 C453 P56460 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 760118 T1672 C453 P56460 oass.SolrIndexSearcher.<init> Opening Searcher@2b48c62a realtime
[junit4:junit4] 2> 760118 T1672 C453 P56460 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760118 T1672 C453 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 760119 T1698 C452 P56463 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 760120 T1698 C452 P56463 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 760123 T1673 C453 P56460 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 760123 T1673 C453 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 760124 T1698 C452 P56463 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 760124 T1698 C452 P56463 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 760124 T1698 C452 P56463 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 760128 T1673 C453 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 760129 T1698 C452 P56463 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 760130 T1698 C452 P56463 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index.20130621143303537
[junit4:junit4] 2> 760131 T1698 C452 P56463 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index.20130621143303537 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3afbf507; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 760135 T1673 C453 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 760137 T1698 C452 P56463 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 760139 T1698 C452 P56463 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 760139 T1698 C452 P56463 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 760144 T1698 C452 P56463 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d4d9bc3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d4d9bc3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 760145 T1698 C452 P56463 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 760145 T1698 C452 P56463 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 760146 T1698 C452 P56463 oass.SolrIndexSearcher.<init> Opening Searcher@3293a9b2 main
[junit4:junit4] 2> 760147 T1697 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3293a9b2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 760148 T1698 C452 P56463 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index.20130621143303537 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index.20130621143303537;done=true>>]
[junit4:junit4] 2> 760148 T1698 C452 P56463 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index.20130621143303537
[junit4:junit4] 2> 760149 T1698 C452 P56463 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index.20130621143303537
[junit4:junit4] 2> 760150 T1698 C452 P56463 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 760150 T1698 C452 P56463 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 760150 T1698 C452 P56463 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 760150 T1698 C452 P56463 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 760152 T1698 C452 P56463 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 760579 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760580 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56467_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56467/md/q"}
[junit4:junit4] 2> 760583 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56463_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56463/md/q"}
[junit4:junit4] 2> 760595 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 760595 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 760595 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 760595 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 760595 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 760595 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 760957 T1637 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56477_md%2Fq
[junit4:junit4] 2> 760959 T1637 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56477_md%2Fq
[junit4:junit4] 2> 760961 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 760961 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 760961 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 760961 T1679 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 760961 T1712 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 760961 T1666 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 760963 T1695 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 760963 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 760962 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 760962 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 760969 T1729 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 760969 T1659 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 760980 T1732 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 760980 T1732 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 761457 T1670 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 761457 T1670 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:56467_md%252Fq&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 762100 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 762101 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56477_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56477/md/q"}
[junit4:junit4] 2> 762101 T1660 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 762102 T1660 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 762121 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 762121 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 762121 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 762121 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 762121 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 762121 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 762983 T1732 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371814382860/collection1
[junit4:junit4] 2> 762983 T1732 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 762984 T1732 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 762984 T1732 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 762986 T1732 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371814382860/collection1/'
[junit4:junit4] 2> 762987 T1732 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1371814382860/collection1/lib/README' to classloader
[junit4:junit4] 2> 762988 T1732 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1371814382860/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 763086 T1732 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 763152 T1732 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 763253 T1732 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 763260 T1732 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> C451_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:56467_md%2Fq, base_url=http://127.0.0.1:56467/md/q}
[junit4:junit4] 2> 763459 T1715 C451 P56467 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:56460/md/q/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 763459 T1715 C451 P56467 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56467/md/q START replicas=[http://127.0.0.1:56460/md/q/collection1/] nUpdates=100
[junit4:junit4] 2> 763460 T1715 C451 P56467 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 763461 T1715 C451 P56467 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 763461 T1715 C451 P56467 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 763461 T1715 C451 P56467 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 763461 T1715 C451 P56467 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 763462 T1715 C451 P56467 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:56460/md/q/collection1/. core=collection1
[junit4:junit4] 2> 763462 T1715 C451 P56467 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 763463 T1671 C453 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 763483 T1674 C453 P56460 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 763504 T1674 C453 P56460 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 763505 T1674 C453 P56460 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 763507 T1674 C453 P56460 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 763508 T1674 C453 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 25
[junit4:junit4] 2> 763509 T1715 C451 P56467 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 763509 T1715 C451 P56467 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 763512 T1673 C453 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 763513 T1715 C451 P56467 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 763514 T1715 C451 P56467 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 763514 T1715 C451 P56467 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 763517 T1673 C453 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1
[junit4:junit4] 2> 763518 T1715 C451 P56467 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 763520 T1715 C451 P56467 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index.20130621143306926
[junit4:junit4] 2> 763521 T1715 C451 P56467 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index.20130621143306926 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4cc2ca34; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 763525 T1673 C453 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 763527 T1715 C451 P56467 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 763531 T1715 C451 P56467 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 763531 T1715 C451 P56467 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 763538 T1715 C451 P56467 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff4edc0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff4edc0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 763539 T1715 C451 P56467 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 763539 T1715 C451 P56467 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 763540 T1715 C451 P56467 oass.SolrIndexSearcher.<init> Opening Searcher@664f2671 main
[junit4:junit4] 2> 763542 T1714 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@664f2671 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 763542 T1715 C451 P56467 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index.20130621143306926 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index.20130621143306926;done=true>>]
[junit4:junit4] 2> 763543 T1715 C451 P56467 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index.20130621143306926
[junit4:junit4] 2> 763543 T1715 C451 P56467 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index.20130621143306926
[junit4:junit4] 2> 763544 T1715 C451 P56467 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 763545 T1715 C451 P56467 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 763545 T1715 C451 P56467 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 763545 T1715 C451 P56467 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 763547 T1715 C451 P56467 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 763625 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 763626 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56467_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56467/md/q"}
[junit4:junit4] 2> 763646 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 763647 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 763646 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 763646 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 763646 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 763646 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 763975 T1732 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 763976 T1732 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 763977 T1732 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 763988 T1732 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 763991 T1732 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 764001 T1732 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 764022 T1732 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 764026 T1732 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 764026 T1732 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 764027 T1732 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 764027 T1732 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 764027 T1732 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 764028 T1732 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 764028 T1732 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 764028 T1732 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371814382860/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/
[junit4:junit4] 2> 764028 T1732 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7692fed2
[junit4:junit4] 2> 764029 T1732 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 764030 T1732 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4
[junit4:junit4] 2> 764031 T1732 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index/
[junit4:junit4] 2> 764032 T1732 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 764033 T1732 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index
[junit4:junit4] 2> 764038 T1732 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a0e4d9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 764039 T1732 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 764042 T1732 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 764042 T1732 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 764042 T1732 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 764043 T1732 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 764043 T1732 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 764044 T1732 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 764047 T1732 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 764048 T1732 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 764048 T1732 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 764067 T1732 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 764078 T1732 oass.SolrIndexSearcher.<init> Opening Searcher@4a0853b4 main
[junit4:junit4] 2> 764079 T1732 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 764079 T1732 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 764082 T1735 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4a0853b4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 764084 T1732 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 764084 T1732 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56477/md/q collection:collection1 shard:shard1
[junit4:junit4] 2> 764087 T1732 oasc.ZkController.register We are http://127.0.0.1:56477/md/q/collection1/ and leader is http://127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 764087 T1732 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56477/md/q
[junit4:junit4] 2> 764087 T1732 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 764087 T1732 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C454 name=collection1 org.apache.solr.core.SolrCore@3dc2168f url=http://127.0.0.1:56477/md/q/collection1 node=127.0.0.1:56477_md%2Fq C454_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:56477_md%2Fq, base_url=http://127.0.0.1:56477/md/q}
[junit4:junit4] 2> 764087 T1736 C454 P56477 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 764088 T1732 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 764089 T1736 C454 P56477 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 764089 T1736 C454 P56477 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 764089 T1736 C454 P56477 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 764089 T1637 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 764090 T1637 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 764090 T1736 C454 P56477 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 764091 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 764110 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 764111 T1675 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 764112 T1637 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 764112 T1637 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 764113 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 765114 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 765151 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 765152 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56477_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56477/md/q"}
[junit4:junit4] 2> 765165 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 765165 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 765165 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 765165 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 765165 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 765165 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 766113 T1675 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 766113 T1675 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:56477_md%252Fq&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 766116 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 767118 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C454_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:56477_md%2Fq, base_url=http://127.0.0.1:56477/md/q}
[junit4:junit4] 2> 768115 T1736 C454 P56477 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:56460/md/q/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 768115 T1736 C454 P56477 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56477/md/q START replicas=[http://127.0.0.1:56460/md/q/collection1/] nUpdates=100
[junit4:junit4] 2> 768116 T1736 C454 P56477 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 768116 T1736 C454 P56477 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 768116 T1736 C454 P56477 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 768116 T1736 C454 P56477 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 768116 T1736 C454 P56477 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 768117 T1736 C454 P56477 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:56460/md/q/collection1/. core=collection1
[junit4:junit4] 2> 768117 T1736 C454 P56477 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C455 name=collection1 org.apache.solr.core.SolrCore@4b21f7dc url=http://127.0.0.1:56460/md/q/collection1 node=127.0.0.1:56460_md%2Fq C455_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56460_md%2Fq, base_url=http://127.0.0.1:56460/md/q, leader=true}
[junit4:junit4] 2> 768118 T1671 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 768120 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 768130 T1669 C455 P56460 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 768160 T1669 C455 P56460 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 768160 T1669 C455 P56460 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 768162 T1669 C455 P56460 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 768162 T1669 C455 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 32
[junit4:junit4] 2> 768164 T1736 C454 P56477 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 768164 T1736 C454 P56477 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 768166 T1673 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 768167 T1736 C454 P56477 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 768167 T1736 C454 P56477 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 768167 T1736 C454 P56477 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 768171 T1673 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1
[junit4:junit4] 2> 768171 T1736 C454 P56477 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 768173 T1736 C454 P56477 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index.20130621143311580
[junit4:junit4] 2> 768174 T1736 C454 P56477 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index.20130621143311580 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b6a5694; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 768178 T1673 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 768180 T1736 C454 P56477 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 768182 T1736 C454 P56477 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 768182 T1736 C454 P56477 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 768187 T1736 C454 P56477 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a0e4d9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a0e4d9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 768188 T1736 C454 P56477 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 768188 T1736 C454 P56477 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 768189 T1736 C454 P56477 oass.SolrIndexSearcher.<init> Opening Searcher@5a658bfa main
[junit4:junit4] 2> 768190 T1735 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a658bfa main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 768191 T1736 C454 P56477 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index.20130621143311580 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index.20130621143311580;done=true>>]
[junit4:junit4] 2> 768191 T1736 C454 P56477 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index.20130621143311580
[junit4:junit4] 2> 768192 T1736 C454 P56477 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index.20130621143311580
[junit4:junit4] 2> 768193 T1736 C454 P56477 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 768193 T1736 C454 P56477 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 768193 T1736 C454 P56477 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 768193 T1736 C454 P56477 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 768195 T1736 C454 P56477 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 769122 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 769673 T1660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 769674 T1660 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56477_md%2Fq",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56477/md/q"}
[junit4:junit4] 2> 769678 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 769678 T1695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 769678 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 769678 T1659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 769678 T1679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 769678 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 770124 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 770125 T1637 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C456 name=collection1 org.apache.solr.core.SolrCore@36777e4 url=http://127.0.0.1:56449/md/q/collection1 node=127.0.0.1:56449_md%2Fq C456_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:56449_md%2Fq, base_url=http://127.0.0.1:56449/md/q, leader=true}
[junit4:junit4] 2> 770139 T1649 C456 P56449 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 770145 T1649 C456 P56449 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ad338e8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 770146 T1649 C456 P56449 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 770151 T1649 C456 P56449 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ad338e8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ad338e8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 770151 T1649 C456 P56449 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 770153 T1649 C456 P56449 oass.SolrIndexSearcher.<init> Opening Searcher@7b8e613d main
[junit4:junit4] 2> 770154 T1649 C456 P56449 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 770155 T1663 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b8e613d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 770155 T1649 C456 P56449 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 16
[junit4:junit4] 2> ASYNC NEW_CORE C457 name=collection1 org.apache.solr.core.SolrCore@1ca0bd7d url=http://127.0.0.1:56467/md/q/collection1 node=127.0.0.1:56467_md%2Fq C457_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56467_md%2Fq, base_url=http://127.0.0.1:56467/md/q}
[junit4:junit4] 2> 770179 T1702 C457 P56467 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:56460/md/q/collection1/, StdNode: http://127.0.0.1:56463/md/q/collection1/, StdNode: http://127.0.0.1:56467/md/q/collection1/, StdNode: http://127.0.0.1:56477/md/q/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 770184 T1672 C455 P56460 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 770184 T1703 C457 P56467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C454_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56477_md%2Fq, base_url=http://127.0.0.1:56477/md/q}
[junit4:junit4] 2> 770199 T1719 C454 P56477 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C458 name=collection1 org.apache.solr.core.SolrCore@4bde63d4 url=http://127.0.0.1:56463/md/q/collection1 node=127.0.0.1:56463_md%2Fq C458_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56463_md%2Fq, base_url=http://127.0.0.1:56463/md/q}
[junit4:junit4] 2> 770199 T1685 C458 P56463 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 770201 T1672 C455 P56460 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 770202 T1672 C455 P56460 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 770204 T1703 C457 P56467 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff4edc0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff4edc0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 770204 T1672 C455 P56460 oass.SolrIndexSearcher.<init> Opening Searcher@5d261873 main
[junit4:junit4] 2> 770204 T1703 C457 P56467 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 770205 T1672 C455 P56460 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 770206 T1681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5d261873 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 770207 T1672 C455 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> 770207 T1703 C457 P56467 oass.SolrIndexSearcher.<init> Opening Searcher@74760a83 main
[junit4:junit4] 2> 770208 T1703 C457 P56467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 770209 T1714 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74760a83 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 770209 T1703 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 25
[junit4:junit4] 2> 770214 T1719 C454 P56477 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a0e4d9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a0e4d9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 770215 T1719 C454 P56477 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 770217 T1719 C454 P56477 oass.SolrIndexSearcher.<init> Opening Searcher@2c392f5 main
[junit4:junit4] 2> 770217 T1719 C454 P56477 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 770218 T1735 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c392f5 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 770219 T1719 C454 P56477 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 770221 T1685 C458 P56463 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d4d9bc3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d4d9bc3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 770222 T1685 C458 P56463 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 770224 T1685 C458 P56463 oass.SolrIndexSearcher.<init> Opening Searcher@1293ed43 main
[junit4:junit4] 2> 770224 T1685 C458 P56463 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 770225 T1697 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1293ed43 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 770225 T1685 C458 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 770226 T1702 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 48
[junit4:junit4] 2> 770227 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 770230 T1670 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 770233 T1686 C458 P56463 oasc.SolrCore.execute [collection1] webapp=/md/q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 770235 T1704 C457 P56467 oasc.SolrCore.execute [collection1] webapp=/md/q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 770237 T1720 C454 P56477 oasc.SolrCore.execute [collection1] webapp=/md/q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 772244 T1649 C456 P56449 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438451651733094400)} 0 3
[junit4:junit4] 2> 772257 T1687 C458 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={update.distrib=FROMLEADER&_version_=-1438451651741483008&update.from=http://127.0.0.1:56460/md/q/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438451651741483008)} 0 2
[junit4:junit4] 2> 772258 T1705 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={update.distrib=FROMLEADER&_version_=-1438451651741483008&update.from=http://127.0.0.1:56460/md/q/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438451651741483008)} 0 3
[junit4:junit4] 2> 772258 T1722 C454 P56477 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={update.distrib=FROMLEADER&_version_=-1438451651741483008&update.from=http://127.0.0.1:56460/md/q/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438451651741483008)} 0 3
[junit4:junit4] 2> 772259 T1674 C455 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438451651741483008)} 0 9
[junit4:junit4] 2> 772260 T1721 C454 P56477 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 13
[junit4:junit4] 2> 772266 T1649 C456 P56449 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={wt=javabin&version=2} {add=[0 (1438451651755114496)]} 0 3
[junit4:junit4] 2> 772276 T1687 C458 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438451651760357376)]} 0 3
[junit4:junit4] 2> 772276 T1722 C454 P56477 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438451651760357376)]} 0 3
[junit4:junit4] 2> 772276 T1705 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438451651760357376)]} 0 3
[junit4:junit4] 2> 772278 T1675 C455 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={wt=javabin&version=2} {add=[0 (1438451651760357376)]} 0 10
[junit4:junit4] 2> 772282 T1649 C456 P56449 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={wt=javabin&version=2} {add=[1 (1438451651773988864)]} 0 1
[junit4:junit4] 2> 772290 T1705 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438451651779231744)]} 0 1
[junit4:junit4] 2> 772290 T1722 C454 P56477 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438451651779231744)]} 0 1
[junit4:junit4] 2> 772290 T1687 C458 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438451651779231744)]} 0 1
[junit4:junit4] 2> 772291 T1669 C455 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56463/md/q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438451651779231744)]} 0 5
[junit4:junit4] 2> 772292 T1688 C458 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={wt=javabin&version=2} {add=[1]} 0 8
[junit4:junit4] 2> 772296 T1649 C456 P56449 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={wt=javabin&version=2} {add=[2 (1438451651788668928)]} 0 1
[junit4:junit4] 2> 772301 T1674 C455 P56460 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:56463/md/q/collection1/]
[junit4:junit4] 2> 772302 T1674 C455 P56460 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:56463/md/q/collection1/ against:[http://127.0.0.1:56463/md/q/collection1/] result:true
[junit4:junit4] 2> 772302 T1674 C455 P56460 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:56467/md/q/collection1/ against:[http://127.0.0.1:56463/md/q/collection1/] result:false
[junit4:junit4] 2> 772303 T1674 C455 P56460 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:56477/md/q/collection1/ against:[http://127.0.0.1:56463/md/q/collection1/] result:false
[junit4:junit4] 2> 772307 T1722 C454 P56477 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:56463/md/q/collection1/&wt=javabin&version=2} {add=[2 (1438451651797057536)]} 0 1
[junit4:junit4] 2> 772307 T1705 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:56463/md/q/collection1/&wt=javabin&version=2} {add=[2 (1438451651797057536)]} 0 1
[junit4:junit4] 2> 772308 T1674 C455 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56477/md/q/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:56463/md/q/collection1/&wt=javabin&version=2} {add=[2 (1438451651797057536)]} 0 7
[junit4:junit4] 2> 772309 T1721 C454 P56477 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={test.distrib.skip.servers=http://127.0.0.1:56463/md/q/collection1/&wt=javabin&version=2} {add=[2]} 0 10
[junit4:junit4] 2> 772312 T1649 C456 P56449 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={wt=javabin&version=2} {add=[3 (1438451651805446144)]} 0 1
[junit4:junit4] 2> 772318 T1672 C455 P56460 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:56463/md/q/collection1/]
[junit4:junit4] 2> 772319 T1672 C455 P56460 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:56463/md/q/collection1/ against:[http://127.0.0.1:56463/md/q/collection1/] result:true
[junit4:junit4] 2> 772319 T1672 C455 P56460 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:56467/md/q/collection1/ against:[http://127.0.0.1:56463/md/q/collection1/] result:false
[junit4:junit4] 2> 772320 T1672 C455 P56460 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:56477/md/q/collection1/ against:[http://127.0.0.1:56463/md/q/collection1/] result:false
[junit4:junit4] 2> 772324 T1722 C454 P56477 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:56463/md/q/collection1/&wt=javabin&version=2} {add=[3 (1438451651814883328)]} 0 1
[junit4:junit4] 2> 772324 T1705 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56460/md/q/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:56463/md/q/collection1/&wt=javabin&version=2} {add=[3 (1438451651814883328)]} 0 1
[junit4:junit4] 2> 772325 T1672 C455 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={distrib.from=http://127.0.0.1:56467/md/q/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:56463/md/q/collection1/&wt=javabin&version=2} {add=[3 (1438451651814883328)]} 0 7
[junit4:junit4] 2> 772326 T1702 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={test.distrib.skip.servers=http://127.0.0.1:56463/md/q/collection1/&test.distrib.skip.servers=http://127.0.0.1:56467/md/q/collection1/&wt=javabin&version=2} {add=[3]} 0 11
[junit4:junit4] 2> 772329 T1649 C456 P56449 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 772393 T1649 C456 P56449 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ad338e8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ad338e8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 772394 T1649 C456 P56449 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 772401 T1649 C456 P56449 oass.SolrIndexSearcher.<init> Opening Searcher@7d5d97d9 main
[junit4:junit4] 2> 772401 T1649 C456 P56449 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 772402 T1663 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d5d97d9 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 772403 T1649 C456 P56449 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 74
[junit4:junit4] 2> 772406 T1702 C457 P56467 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:56460/md/q/collection1/, StdNode: http://127.0.0.1:56463/md/q/collection1/, StdNode: http://127.0.0.1:56467/md/q/collection1/, StdNode: http://127.0.0.1:56477/md/q/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 772409 T1672 C455 P56460 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 772409 T1685 C458 P56463 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 772410 T1703 C457 P56467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 772410 T1719 C454 P56477 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 772506 T1685 C458 P56463 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d4d9bc3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d4d9bc3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 772507 T1685 C458 P56463 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 772512 T1703 C457 P56467 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff4edc0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff4edc0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 772512 T1672 C455 P56460 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4742d356; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 772512 T1719 C454 P56477 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a0e4d9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a0e4d9; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 772513 T1685 C458 P56463 oass.SolrIndexSearcher.<init> Opening Searcher@684c5129 main
[junit4:junit4] 2> 772513 T1672 C455 P56460 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 772512 T1703 C457 P56467 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 772514 T1685 C458 P56463 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 772514 T1719 C454 P56477 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 772515 T1697 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@684c5129 main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):C2)}
[junit4:junit4] 2> 772516 T1685 C458 P56463 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 107
[junit4:junit4] 2> 772523 T1703 C457 P56467 oass.SolrIndexSearcher.<init> Opening Searcher@62563e93 main
[junit4:junit4] 2> 772524 T1719 C454 P56477 oass.SolrIndexSearcher.<init> Opening Searcher@65a432a7 main
[junit4:junit4] 2> 772524 T1703 C457 P56467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 772523 T1672 C455 P56460 oass.SolrIndexSearcher.<init> Opening Searcher@728899a1 main
[junit4:junit4] 2> 772525 T1714 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@62563e93 main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 772524 T1719 C454 P56477 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 772525 T1703 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 115
[junit4:junit4] 2> 772525 T1672 C455 P56460 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 772525 T1735 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@65a432a7 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 772526 T1681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@728899a1 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 772527 T1719 C454 P56477 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 117
[junit4:junit4] 2> 772527 T1672 C455 P56460 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 118
[junit4:junit4] 2> 772528 T1702 C457 P56467 oasup.LogUpdateProcessor.finish [collection1] webapp=/md/q path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 122
[junit4:junit4] 2> 772529 T1637 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 772530 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 772531 T1637 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 772533 T1670 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 772536 T1686 C458 P56463 oasc.SolrCore.execute [collection1] webapp=/md/q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 772540 T1704 C457 P56467 oasc.SolrCore.execute [collection1] webapp=/md/q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 772543 T1720 C454 P56477 oasc.SolrCore.execute [collection1] webapp=/md/q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 772544 T1637 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 772564 T1706 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 772565 T1706 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 772580 T1749 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 772580 T1749 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 772581 T1749 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56460/md/q START replicas=[http://127.0.0.1:56463/md/q/collection1/, http://127.0.0.1:56467/md/q/collection1/, http://127.0.0.1:56477/md/q/collection1/] nUpdates=100
[junit4:junit4] 2> 772585 T1689 C458 P56463 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 772585 T1723 C454 P56477 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 772586 T1749 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56460/md/q Received 3 versions from 127.0.0.1:56463/md/q/collection1/
[junit4:junit4] 2> 772585 T1707 C457 P56467 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 772586 T1749 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56460/md/q Our versions are newer. ourLowThreshold=1438451651741483008 otherHigh=1438451651779231744
[junit4:junit4] 2> 772587 T1749 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56460/md/q Received 5 versions from 127.0.0.1:56477/md/q/collection1/
[junit4:junit4] 2> 772587 T1749 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56460/md/q Our versions are newer. ourLowThreshold=1438451651741483008 otherHigh=1438451651797057536
[junit4:junit4] 2> 772588 T1749 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56460/md/q Received 5 versions from 127.0.0.1:56467/md/q/collection1/
[junit4:junit4] 2> 772588 T1749 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56460/md/q Our versions are newer. ourLowThreshold=1438451651741483008 otherHigh=1438451651797057536
[junit4:junit4] 2> 772589 T1749 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56460/md/q DONE. sync succeeded
[junit4:junit4] 2> 772589 T1749 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 772589 T1749 oasc.SyncStrategy.syncToMe http://127.0.0.1:56460/md/q/collection1/: try and ask http://127.0.0.1:56463/md/q/collection1/ to sync
[junit4:junit4] 2> 772590 T1749 oasc.SyncStrategy.syncToMe http://127.0.0.1:56460/md/q/collection1/: try and ask http://127.0.0.1:56467/md/q/collection1/ to sync
[junit4:junit4] 2> 772590 T1749 oasc.SyncStrategy.syncToMe http://127.0.0.1:56460/md/q/collection1/: try and ask http://127.0.0.1:56477/md/q/collection1/ to sync
[junit4:junit4] 2> 772593 T1690 C458 P56463 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56463/md/q START replicas=[http://127.0.0.1:56460/md/q/collection1/] nUpdates=100
[junit4:junit4] 2> 772593 T1708 C457 P56467 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56467/md/q START replicas=[http://127.0.0.1:56460/md/q/collection1/] nUpdates=100
[junit4:junit4] 2> 772594 T1724 C454 P56477 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56477/md/q START replicas=[http://127.0.0.1:56460/md/q/collection1/] nUpdates=100
[junit4:junit4] 2> 772595 T1759 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 772596 T1760 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 772596 T1708 C457 P56467 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56467/md/q Received 5 versions from 127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 772597 T1724 C454 P56477 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56477/md/q Received 5 versions from 127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 772596 T1671 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 772597 T1724 C454 P56477 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56477/md/q Our versions are newer. ourLowThreshold=1438451651741483008 otherHigh=1438451651797057536
[junit4:junit4] 2> 772597 T1708 C457 P56467 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56467/md/q Our versions are newer. ourLowThreshold=1438451651741483008 otherHigh=1438451651797057536
[junit4:junit4] 2> 772598 T1724 C454 P56477 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56477/md/q DONE. sync succeeded
[junit4:junit4] 2> 772598 T1690 C458 P56463 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56463/md/q Received 5 versions from 127.0.0.1:56460/md/q/collection1/
[junit4:junit4] 2> 772599 T1724 C454 P56477 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={sync=http://127.0.0.1:56460/md/q/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=5
[junit4:junit4] 2> 772598 T1708 C457 P56467 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56467/md/q DONE. sync succeeded
[junit4:junit4] 2> 772599 T1690 C458 P56463 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:56463/md/q Requesting updates from 127.0.0.1:56460/md/q/collection1/n=2 versions=[1438451651814883328, 1438451651797057536]
[junit4:junit4] 2> 772600 T1749 oasc.SyncStrategy.syncToMe http://127.0.0.1:56460/md/q/collection1/: sync completed with http://127.0.0.1:56477/md/q/collection1/
[junit4:junit4] 2> 772600 T1708 C457 P56467 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={sync=http://127.0.0.1:56460/md/q/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=7
[junit4:junit4] 2> 772601 T1749 oasc.SyncStrategy.syncToMe http://127.0.0.1:56460/md/q/collection1/: sync completed with http://127.0.0.1:56467/md/q/collection1/
[junit4:junit4] 2> 772603 T1671 C455 P56460 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={distrib=false&getUpdates=1438451651814883328,1438451651797057536&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 772609 T1690 C458 P56463 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438451651797057536), 3 (1438451651814883328)]} 0 5
[junit4:junit4] 2> 772609 T1690 C458 P56463 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56463/md/q DONE. sync succeeded
[junit4:junit4] 2> 772610 T1690 C458 P56463 oasc.SolrCore.execute [collection1] webapp=/md/q path=/get params={sync=http://127.0.0.1:56460/md/q/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=18
[junit4:junit4] 2> 772610 T1749 oasc.SyncStrategy.syncToMe http://127.0.0.1:56460/md/q/collection1/: sync completed with http://127.0.0.1:56463/md/q/collection1/
[junit4:junit4] 2> 772611 T1749 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=31
[junit4:junit4] 2> 772612 T1706 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=48
[junit4:junit4] 2> 772612 T1637 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 772613 T1637 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 772614 T1637 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 772614 T1637 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 772617 T1649 C456 P56449 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 772636 T1649 C456 P56449 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1371814364375/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ad338e8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-137181
[...truncated too long message...]
ion1",
[junit4:junit4] 1> "node_name":"127.0.0.1:56467_md%2Fq",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:56467/md/q",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "5":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:56477_md%2Fq",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:56477/md/q"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:56449_md%2Fq",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:56449/md/q",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:56467_md%2Fq (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:56460_md%2Fq (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:56477_md%2Fq (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:56449_md%2Fq (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89903226184204302-2-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89903226184204298-4-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89903226184204300-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:56467_md%2Fq",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:56467/md/q"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89903226184204291-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:56449_md%2Fq",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:56449/md/q"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89903226184204291-127.0.0.1:56449_md%2Fq-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89903226184204302-127.0.0.1:56460_md%2Fq-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89903226184204298-127.0.0.1:56467_md%2Fq-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89903226184204300-127.0.0.1:56477_md%2Fq-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89903226184204291-127.0.0.1:56449_md%2Fq-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=4FBBD6639EBFF2F4 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_IQ -Dtests.timezone=Europe/Riga -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE 523s J1 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:56460/md/q/collection1lastClient and got 257 from http://127.0.0.1:56467/md/q/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([4FBBD6639EBFF2F4:CE5D587BE9E092C8]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 1263505 T1637 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 522549 T1636 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_IQ, timezone=Europe/Riga
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=216733232,total=490733568
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestCollationKeyRangeQueries, TestIBSimilarityFactory, BadIndexSchemaTest, SliceStateTest, TestReversedWildcardFilterFactory, AnalysisAfterCoreReloadTest, PluginInfoTest, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, SliceStateUpdateTest, QueryElevationComponentTest, LoggingHandlerTest, TestFunctionQuery, TestWriterPerf, UniqFieldsUpdateProcessorFactoryTest, ResourceLoaderTest, NotRequiredUniqueKeyTest, MoreLikeThisHandlerTest, TestSolrQueryParser, TestMergePolicyConfig, FileUtilsTest, TestIndexSearcher, SolrPluginUtilsTest, NoCacheHeaderTest, FastVectorHighlighterTest, IndexSchemaTest, TestCloudManagedSchemaAddField, SuggesterTSTTest, IndexReaderFactoryTest, AlternateDirectoryTest, TestFoldingMultitermQuery, TestExtendedDismaxParser, StandardRequestHandlerTest, SpellPossibilityIteratorTest, XsltUpdateRequestHandlerTest, ChaosMonkeyNothingIsSafeTest, TestLFUCache, TestJmxMonitoredMap, ClusterStateUpdateTest, OverseerTest, SpellCheckCollatorTest, SpellingQueryConverterTest, TestRangeQuery, ZkControllerTest, TestFaceting, SchemaVersionSpecificBehaviorTest, OpenExchangeRatesOrgProviderTest, ReturnFieldsTest, CurrencyFieldXmlFileTest, TermVectorComponentTest, TestFastWriter, TestCoreContainer, TestDefaultSimilarityFactory, TestConfig, TestSurroundQueryParser, TestQuerySenderNoQuery, TestPerFieldSimilarity, TestRandomDVFaceting, TestPseudoReturnFields, DebugComponentTest, ZkCLITest, SpatialFilterTest, TestFieldCollectionResource, SearchHandlerTest, TestGroupingSearch, SystemInfoHandlerTest, TestComponentsName, TestManagedSchemaFieldResource, TestSerializedLuceneMatchVersion, TestRealTimeGet, OverseerCollectionProcessorTest, SampleTest, ShardSplitTest, SyncSliceTest]
[junit4:junit4] Completed on J1 in 522.84s, 1 test, 1 failure <<< FAILURES!
[...truncated 466 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:392: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:372: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1246: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:890: There were test failures: 298 suites, 1254 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 52 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure