You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/06/28 16:09:10 UTC
[JENKINS] Lucene-Solr-Tests-4.x-Java6 - Build # 1743 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java6/1743/
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:57573/imocl/r/collection1lastClient and got 264 from http://127.0.0.1:30511/imocl/r/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:57573/imocl/r/collection1lastClient and got 264 from http://127.0.0.1:30511/imocl/r/collection1
at __randomizedtesting.SeedInfo.seed([A8E99C6F0262BE2A:290F1277753DDE16]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:679)
Build Log:
[...truncated 9130 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 760762 T2643 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /imocl/r
[junit4:junit4] 2> 760767 T2643 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1372427105721
[junit4:junit4] 2> 760768 T2643 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 760769 T2644 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 760870 T2643 oasc.ZkTestServer.run start zk server on port:36425
[junit4:junit4] 2> 760871 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 760875 T2650 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2dfdd4ce name:ZooKeeperConnection Watcher:127.0.0.1:36425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 760875 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 760876 T2643 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 760889 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 760891 T2652 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40d481f9 name:ZooKeeperConnection Watcher:127.0.0.1:36425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 760891 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 760892 T2643 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 760902 T2643 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 760904 T2643 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 760906 T2643 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 760916 T2643 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 760917 T2643 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 760928 T2643 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 760929 T2643 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 761034 T2643 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 761035 T2643 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 761038 T2643 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 761038 T2643 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 761048 T2643 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 761049 T2643 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 761053 T2643 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 761054 T2643 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 761057 T2643 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 761058 T2643 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 761062 T2643 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 761063 T2643 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 761066 T2643 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 761067 T2643 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 761371 T2643 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 761377 T2643 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:29692
[junit4:junit4] 2> 761377 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 761378 T2643 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 761378 T2643 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372427106024
[junit4:junit4] 2> 761379 T2643 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372427106024/solr.xml
[junit4:junit4] 2> 761379 T2643 oasc.CoreContainer.<init> New CoreContainer 152438623
[junit4:junit4] 2> 761380 T2643 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372427106024/'
[junit4:junit4] 2> 761380 T2643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372427106024/'
[junit4:junit4] 2> 761487 T2643 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 761488 T2643 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 761489 T2643 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 761489 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 761490 T2643 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 761490 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 761491 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 761491 T2643 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 761491 T2643 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 761492 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 761507 T2643 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 761508 T2643 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36425/solr
[junit4:junit4] 2> 761508 T2643 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 761509 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 761534 T2663 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26046c5e name:ZooKeeperConnection Watcher:127.0.0.1:36425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 761534 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 761575 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 761586 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 761633 T2665 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e17d2f4 name:ZooKeeperConnection Watcher:127.0.0.1:36425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 761634 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 761661 T2643 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 761753 T2643 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 761758 T2643 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 761773 T2643 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:29692_imocl%2Fr
[junit4:junit4] 2> 761781 T2643 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:29692_imocl%2Fr
[junit4:junit4] 2> 761790 T2643 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 761803 T2643 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 761805 T2643 oasc.Overseer.start Overseer (id=89943382801055747-127.0.0.1:29692_imocl%2Fr-n_0000000000) starting
[junit4:junit4] 2> 761808 T2643 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 761812 T2667 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 761812 T2643 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 761814 T2643 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 761816 T2643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 761818 T2666 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 761822 T2668 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 761822 T2668 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 763321 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 763322 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:29692_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:29692/imocl/r"}
[junit4:junit4] 2> 763322 T2666 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 763323 T2666 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 763334 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 763825 T2668 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372427106024/collection1
[junit4:junit4] 2> 763825 T2668 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 763826 T2668 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 763826 T2668 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 763827 T2668 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372427106024/collection1/'
[junit4:junit4] 2> 763829 T2668 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372427106024/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 763829 T2668 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372427106024/collection1/lib/README' to classloader
[junit4:junit4] 2> 763878 T2668 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 763935 T2668 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 764036 T2668 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 764043 T2668 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 764632 T2668 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 764633 T2668 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 764633 T2668 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 764642 T2668 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 764645 T2668 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 764656 T2668 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 764660 T2668 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 764663 T2668 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 764664 T2668 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 764664 T2668 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 764664 T2668 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 764665 T2668 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 764665 T2668 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 764665 T2668 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 764666 T2668 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372427106024/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372427105720/control/data/
[junit4:junit4] 2> 764666 T2668 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@408fbecf
[junit4:junit4] 2> 764666 T2668 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 764667 T2668 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/control/data
[junit4:junit4] 2> 764668 T2668 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372427105720/control/data/index/
[junit4:junit4] 2> 764668 T2668 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372427105720/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 764669 T2668 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/control/data/index
[junit4:junit4] 2> 764673 T2668 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 764674 T2668 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 764676 T2668 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 764676 T2668 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 764676 T2668 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 764677 T2668 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 764677 T2668 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 764677 T2668 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 764678 T2668 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 764678 T2668 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 764678 T2668 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 764684 T2668 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 764690 T2668 oass.SolrIndexSearcher.<init> Opening Searcher@1fef023c main
[junit4:junit4] 2> 764691 T2668 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 764691 T2668 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 764694 T2669 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1fef023c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 764695 T2668 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 764695 T2668 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:29692/imocl/r collection:control_collection shard:shard1
[junit4:junit4] 2> 764696 T2668 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 764711 T2668 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 764713 T2668 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 764713 T2668 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 764713 T2668 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:29692/imocl/r/collection1/
[junit4:junit4] 2> 764714 T2668 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 764714 T2668 oasc.SyncStrategy.syncToMe http://127.0.0.1:29692/imocl/r/collection1/ has no replicas
[junit4:junit4] 2> 764714 T2668 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:29692/imocl/r/collection1/
[junit4:junit4] 2> 764714 T2668 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 764838 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 764854 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 764886 T2668 oasc.ZkController.register We are http://127.0.0.1:29692/imocl/r/collection1/ and leader is http://127.0.0.1:29692/imocl/r/collection1/
[junit4:junit4] 2> 764886 T2668 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:29692/imocl/r
[junit4:junit4] 2> 764886 T2668 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 764886 T2668 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 764887 T2668 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 764888 T2668 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 764889 T2643 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0
[junit4:junit4] 2> 764889 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 764890 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 764901 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 764902 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 764904 T2672 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@199bdabd name:ZooKeeperConnection Watcher:127.0.0.1:36425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 764904 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 764905 T2643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 764907 T2643 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 765169 T2643 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 765172 T2643 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57573
[junit4:junit4] 2> 765172 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 765173 T2643 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 765173 T2643 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372427109860
[junit4:junit4] 2> 765174 T2643 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372427109860/solr.xml
[junit4:junit4] 2> 765174 T2643 oasc.CoreContainer.<init> New CoreContainer 341553942
[junit4:junit4] 2> 765175 T2643 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372427109860/'
[junit4:junit4] 2> 765176 T2643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372427109860/'
[junit4:junit4] 2> 765286 T2643 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 765287 T2643 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 765287 T2643 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 765288 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 765288 T2643 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 765289 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 765289 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 765290 T2643 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 765290 T2643 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 765291 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 765307 T2643 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 765308 T2643 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36425/solr
[junit4:junit4] 2> 765309 T2643 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 765310 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 765312 T2683 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f81cc8 name:ZooKeeperConnection Watcher:127.0.0.1:36425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 765312 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 765323 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 765330 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 765332 T2685 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15808c49 name:ZooKeeperConnection Watcher:127.0.0.1:36425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 765332 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 765341 T2643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 766344 T2643 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57573_imocl%2Fr
[junit4:junit4] 2> 766349 T2643 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57573_imocl%2Fr
[junit4:junit4] 2> 766446 T2685 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 766446 T2672 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 766446 T2665 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 766447 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 766449 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 766450 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:29692_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:29692/imocl/r"}
[junit4:junit4] 2> 766453 T2686 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 766453 T2686 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 766560 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57573_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57573/imocl/r"}
[junit4:junit4] 2> 766560 T2666 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 766560 T2666 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 766565 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 766565 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 766619 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 767537 T2686 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372427109860/collection1
[junit4:junit4] 2> 767537 T2686 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 767538 T2686 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 767539 T2686 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 767540 T2686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372427109860/collection1/'
[junit4:junit4] 2> 767542 T2686 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372427109860/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 767542 T2686 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372427109860/collection1/lib/README' to classloader
[junit4:junit4] 2> 767594 T2686 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 767661 T2686 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 767762 T2686 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 767769 T2686 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 768353 T2686 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 768353 T2686 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 768354 T2686 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 768368 T2686 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 768372 T2686 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 768385 T2686 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 768391 T2686 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 768395 T2686 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 768396 T2686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 768396 T2686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 768397 T2686 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 768397 T2686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 768398 T2686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 768398 T2686 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 768398 T2686 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372427109860/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/
[junit4:junit4] 2> 768399 T2686 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@408fbecf
[junit4:junit4] 2> 768399 T2686 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 768401 T2686 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1
[junit4:junit4] 2> 768401 T2686 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index/
[junit4:junit4] 2> 768402 T2686 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 768403 T2686 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index
[junit4:junit4] 2> 768434 T2686 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 768434 T2686 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 768437 T2686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 768438 T2686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 768438 T2686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 768439 T2686 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 768440 T2686 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 768440 T2686 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 768440 T2686 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 768441 T2686 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 768442 T2686 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 768451 T2686 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 768460 T2686 oass.SolrIndexSearcher.<init> Opening Searcher@2dc2b27a main
[junit4:junit4] 2> 768461 T2686 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 768462 T2686 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 768467 T2687 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2dc2b27a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 768469 T2686 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 768469 T2686 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57573/imocl/r collection:collection1 shard:shard1
[junit4:junit4] 2> 768470 T2686 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 768485 T2686 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 768494 T2686 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 768494 T2686 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 768494 T2686 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 768495 T2686 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 768495 T2686 oasc.SyncStrategy.syncToMe http://127.0.0.1:57573/imocl/r/collection1/ has no replicas
[junit4:junit4] 2> 768495 T2686 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 768495 T2686 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 769623 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 769644 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 769644 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 769644 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 769682 T2686 oasc.ZkController.register We are http://127.0.0.1:57573/imocl/r/collection1/ and leader is http://127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 769682 T2686 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57573/imocl/r
[junit4:junit4] 2> 769682 T2686 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 769682 T2686 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 769683 T2686 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 769685 T2686 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 769686 T2643 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0
[junit4:junit4] 2> 769686 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 769687 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 769959 T2643 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 769962 T2643 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:40244
[junit4:junit4] 2> 769963 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 769963 T2643 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 769964 T2643 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372427114655
[junit4:junit4] 2> 769964 T2643 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372427114655/solr.xml
[junit4:junit4] 2> 769965 T2643 oasc.CoreContainer.<init> New CoreContainer 300264882
[junit4:junit4] 2> 769965 T2643 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372427114655/'
[junit4:junit4] 2> 769966 T2643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372427114655/'
[junit4:junit4] 2> 770074 T2643 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 770075 T2643 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 770075 T2643 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 770076 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 770076 T2643 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 770077 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 770077 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 770078 T2643 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 770078 T2643 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 770079 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 770093 T2643 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 770094 T2643 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36425/solr
[junit4:junit4] 2> 770095 T2643 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 770096 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 770098 T2699 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37b98284 name:ZooKeeperConnection Watcher:127.0.0.1:36425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 770098 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 770100 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 770110 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 770112 T2701 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d8e7ce6 name:ZooKeeperConnection Watcher:127.0.0.1:36425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 770113 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 770117 T2643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 771120 T2643 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40244_imocl%2Fr
[junit4:junit4] 2> 771125 T2643 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40244_imocl%2Fr
[junit4:junit4] 2> 771127 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 771127 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 771127 T2665 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 771127 T2701 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 771128 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 771128 T2672 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 771138 T2685 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 771143 T2702 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 771143 T2702 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 771150 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 771151 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57573_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57573/imocl/r"}
[junit4:junit4] 2> 771154 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:40244_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40244/imocl/r"}
[junit4:junit4] 2> 771154 T2666 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 771155 T2666 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 771221 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 771221 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 771221 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 771221 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 772146 T2702 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372427114655/collection1
[junit4:junit4] 2> 772146 T2702 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 772147 T2702 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 772147 T2702 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 772148 T2702 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372427114655/collection1/'
[junit4:junit4] 2> 772150 T2702 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372427114655/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 772150 T2702 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372427114655/collection1/lib/README' to classloader
[junit4:junit4] 2> 772206 T2702 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 772263 T2702 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 772364 T2702 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 772371 T2702 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 773028 T2702 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 773029 T2702 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 773029 T2702 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 773044 T2702 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 773049 T2702 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 773062 T2702 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 773067 T2702 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 773072 T2702 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 773073 T2702 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 773073 T2702 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 773073 T2702 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 773074 T2702 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 773075 T2702 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 773075 T2702 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 773075 T2702 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372427114655/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/
[junit4:junit4] 2> 773075 T2702 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@408fbecf
[junit4:junit4] 2> 773076 T2702 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 773077 T2702 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2
[junit4:junit4] 2> 773078 T2702 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/index/
[junit4:junit4] 2> 773079 T2702 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 773080 T2702 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/index
[junit4:junit4] 2> 773086 T2702 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 773086 T2702 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 773089 T2702 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 773089 T2702 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 773090 T2702 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 773091 T2702 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 773092 T2702 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 773092 T2702 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 773092 T2702 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 773093 T2702 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 773093 T2702 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 773104 T2702 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 773114 T2702 oass.SolrIndexSearcher.<init> Opening Searcher@5b0b28df main
[junit4:junit4] 2> 773115 T2702 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 773115 T2702 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 773120 T2703 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5b0b28df main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 773123 T2702 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 773123 T2702 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40244/imocl/r collection:collection1 shard:shard1
[junit4:junit4] 2> 773126 T2702 oasc.ZkController.register We are http://127.0.0.1:40244/imocl/r/collection1/ and leader is http://127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 773126 T2702 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40244/imocl/r
[junit4:junit4] 2> 773126 T2702 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 773127 T2702 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C268 name=collection1 org.apache.solr.core.SolrCore@732620b url=http://127.0.0.1:40244/imocl/r/collection1 node=127.0.0.1:40244_imocl%2Fr C268_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:40244_imocl%2Fr, base_url=http://127.0.0.1:40244/imocl/r}
[junit4:junit4] 2> 773127 T2704 C268 P40244 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 773127 T2702 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 773128 T2704 C268 P40244 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 773129 T2704 C268 P40244 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 773129 T2704 C268 P40244 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 773129 T2643 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0
[junit4:junit4] 2> 773130 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 773131 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 773136 T2704 C268 P40244 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 773153 T2675 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 773434 T2643 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 773436 T2643 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:30511
[junit4:junit4] 2> 773437 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 773437 T2643 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 773438 T2643 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372427118104
[junit4:junit4] 2> 773438 T2643 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372427118104/solr.xml
[junit4:junit4] 2> 773438 T2643 oasc.CoreContainer.<init> New CoreContainer 1695016755
[junit4:junit4] 2> 773439 T2643 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372427118104/'
[junit4:junit4] 2> 773439 T2643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372427118104/'
[junit4:junit4] 2> 773544 T2643 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 773545 T2643 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 773545 T2643 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 773546 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 773546 T2643 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 773547 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 773547 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 773548 T2643 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 773548 T2643 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 773549 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 773564 T2643 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 773565 T2643 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36425/solr
[junit4:junit4] 2> 773565 T2643 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 773566 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 773569 T2716 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68be49a5 name:ZooKeeperConnection Watcher:127.0.0.1:36425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 773569 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 773579 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 773590 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 773592 T2718 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@331824ce name:ZooKeeperConnection Watcher:127.0.0.1:36425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 773592 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 773604 T2643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 774251 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 774252 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:40244_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40244/imocl/r"}
[junit4:junit4] 2> 774267 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 774267 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 774267 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 774267 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 774267 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 774607 T2643 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:30511_imocl%2Fr
[junit4:junit4] 2> 774609 T2643 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:30511_imocl%2Fr
[junit4:junit4] 2> 774611 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 774611 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 774611 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 774611 T2665 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 774611 T2718 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 774612 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 774612 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 774612 T2701 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 774620 T2685 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 774621 T2672 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 774625 T2719 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 774625 T2719 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 775155 T2675 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 775155 T2675 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=3&state=recovering&nodeName=127.0.0.1:40244_imocl%252Fr&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 775771 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 775772 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:30511_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:30511/imocl/r"}
[junit4:junit4] 2> 775772 T2666 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 775772 T2666 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 775811 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 775811 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 775811 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 775811 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 775811 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 776629 T2719 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372427118104/collection1
[junit4:junit4] 2> 776629 T2719 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 776630 T2719 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 776630 T2719 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 776632 T2719 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372427118104/collection1/'
[junit4:junit4] 2> 776633 T2719 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372427118104/collection1/lib/README' to classloader
[junit4:junit4] 2> 776634 T2719 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372427118104/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 776684 T2719 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 776741 T2719 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 776842 T2719 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 776849 T2719 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C269 name=collection1 org.apache.solr.core.SolrCore@732620b url=http://127.0.0.1:40244/imocl/r/collection1 node=127.0.0.1:40244_imocl%2Fr C269_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:40244_imocl%2Fr, base_url=http://127.0.0.1:40244/imocl/r}
[junit4:junit4] 2> 777157 T2704 C269 P40244 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57573/imocl/r/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 777157 T2704 C269 P40244 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r START replicas=[http://127.0.0.1:57573/imocl/r/collection1/] nUpdates=100
[junit4:junit4] 2> 777158 T2704 C269 P40244 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 777158 T2704 C269 P40244 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 777158 T2704 C269 P40244 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 777158 T2704 C269 P40244 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 777158 T2704 C269 P40244 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 777159 T2704 C269 P40244 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57573/imocl/r/collection1/. core=collection1
[junit4:junit4] 2> 777159 T2704 C269 P40244 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C270 name=collection1 org.apache.solr.core.SolrCore@6a3af151 url=http://127.0.0.1:57573/imocl/r/collection1 node=127.0.0.1:57573_imocl%2Fr C270_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57573_imocl%2Fr, base_url=http://127.0.0.1:57573/imocl/r, leader=true}
[junit4:junit4] 2> 777167 T2676 C270 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 777172 T2677 C270 P57573 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 777177 T2677 C270 P57573 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 777178 T2677 C270 P57573 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 777179 T2677 C270 P57573 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 777179 T2677 C270 P57573 oass.SolrIndexSearcher.<init> Opening Searcher@a1172c5 realtime
[junit4:junit4] 2> 777180 T2677 C270 P57573 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 777180 T2677 C270 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 777181 T2704 C269 P40244 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 777181 T2704 C269 P40244 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 777184 T2678 C270 P57573 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 777185 T2678 C270 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 777185 T2704 C269 P40244 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 777186 T2704 C269 P40244 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 777186 T2704 C269 P40244 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 777186 T2704 C269 P40244 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 777190 T2704 C269 P40244 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 777315 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 777316 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:40244_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:40244/imocl/r"}
[junit4:junit4] 2> 777354 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 777355 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 777355 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 777354 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 777354 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 777472 T2719 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 777473 T2719 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 777473 T2719 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 777488 T2719 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 777492 T2719 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 777505 T2719 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 777511 T2719 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 777515 T2719 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 777516 T2719 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 777516 T2719 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 777516 T2719 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 777517 T2719 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 777518 T2719 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 777518 T2719 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 777518 T2719 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372427118104/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty3/
[junit4:junit4] 2> 777519 T2719 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@408fbecf
[junit4:junit4] 2> 777519 T2719 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 777520 T2719 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty3
[junit4:junit4] 2> 777521 T2719 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty3/index/
[junit4:junit4] 2> 777522 T2719 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 777523 T2719 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty3/index
[junit4:junit4] 2> 777528 T2719 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 777529 T2719 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 777532 T2719 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 777532 T2719 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 777533 T2719 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 777534 T2719 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 777535 T2719 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 777535 T2719 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 777538 T2719 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 777539 T2719 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 777540 T2719 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 777550 T2719 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 777559 T2719 oass.SolrIndexSearcher.<init> Opening Searcher@ecb57e0 main
[junit4:junit4] 2> 777561 T2719 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 777561 T2719 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 777566 T2721 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ecb57e0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 777569 T2719 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 777569 T2719 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:30511/imocl/r collection:collection1 shard:shard1
[junit4:junit4] 2> 777572 T2719 oasc.ZkController.register We are http://127.0.0.1:30511/imocl/r/collection1/ and leader is http://127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 777573 T2719 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:30511/imocl/r
[junit4:junit4] 2> 777573 T2719 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 777573 T2719 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C271 name=collection1 org.apache.solr.core.SolrCore@673e47ea url=http://127.0.0.1:30511/imocl/r/collection1 node=127.0.0.1:30511_imocl%2Fr C271_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:30511_imocl%2Fr, base_url=http://127.0.0.1:30511/imocl/r}
[junit4:junit4] 2> 777574 T2722 C271 P30511 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 777574 T2719 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 777574 T2722 C271 P30511 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 777575 T2722 C271 P30511 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 777575 T2722 C271 P30511 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 777575 T2643 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0
[junit4:junit4] 2> 777576 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 777577 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 777582 T2722 C271 P30511 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 777600 T2679 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 777896 T2643 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 777898 T2643 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62504
[junit4:junit4] 2> 777899 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 777899 T2643 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 777900 T2643 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372427122551
[junit4:junit4] 2> 777900 T2643 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372427122551/solr.xml
[junit4:junit4] 2> 777901 T2643 oasc.CoreContainer.<init> New CoreContainer 1859349859
[junit4:junit4] 2> 777901 T2643 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372427122551/'
[junit4:junit4] 2> 777902 T2643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372427122551/'
[junit4:junit4] 2> 778015 T2643 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 778015 T2643 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 778016 T2643 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 778016 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 778017 T2643 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 778017 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 778018 T2643 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 778018 T2643 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 778019 T2643 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 778019 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 778036 T2643 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 778037 T2643 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36425/solr
[junit4:junit4] 2> 778037 T2643 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 778038 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 778041 T2734 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b285262 name:ZooKeeperConnection Watcher:127.0.0.1:36425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 778041 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 778043 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 778055 T2643 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 778057 T2736 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23500517 name:ZooKeeperConnection Watcher:127.0.0.1:36425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 778057 T2643 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 778068 T2643 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 778858 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 778859 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:30511_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:30511/imocl/r"}
[junit4:junit4] 2> 778879 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 778879 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 778879 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 778879 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 778879 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 778879 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 779071 T2643 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62504_imocl%2Fr
[junit4:junit4] 2> 779073 T2643 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62504_imocl%2Fr
[junit4:junit4] 2> 779075 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 779075 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 779075 T2718 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 779075 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 779075 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 779076 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 779075 T2665 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 779076 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 779076 T2701 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 779077 T2736 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 779078 T2672 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 779078 T2685 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 779082 T2737 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 779082 T2737 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 779602 T2679 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 779602 T2679 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:30511_imocl%252Fr&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 780383 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 780384 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62504_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62504/imocl/r"}
[junit4:junit4] 2> 780384 T2666 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 780384 T2666 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 780427 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 780427 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 780427 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 780427 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 780427 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 780427 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 781085 T2737 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372427122551/collection1
[junit4:junit4] 2> 781085 T2737 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 781086 T2737 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 781086 T2737 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 781088 T2737 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372427122551/collection1/'
[junit4:junit4] 2> 781089 T2737 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372427122551/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 781089 T2737 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372427122551/collection1/lib/README' to classloader
[junit4:junit4] 2> 781139 T2737 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 781205 T2737 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 781306 T2737 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 781312 T2737 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C272 name=collection1 org.apache.solr.core.SolrCore@673e47ea url=http://127.0.0.1:30511/imocl/r/collection1 node=127.0.0.1:30511_imocl%2Fr C272_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:30511_imocl%2Fr, base_url=http://127.0.0.1:30511/imocl/r}
[junit4:junit4] 2> 781604 T2722 C272 P30511 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57573/imocl/r/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 781604 T2722 C272 P30511 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:30511/imocl/r START replicas=[http://127.0.0.1:57573/imocl/r/collection1/] nUpdates=100
[junit4:junit4] 2> 781605 T2722 C272 P30511 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 781605 T2722 C272 P30511 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 781605 T2722 C272 P30511 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 781605 T2722 C272 P30511 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 781605 T2722 C272 P30511 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 781606 T2722 C272 P30511 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57573/imocl/r/collection1/. core=collection1
[junit4:junit4] 2> 781606 T2722 C272 P30511 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C273 name=collection1 org.apache.solr.core.SolrCore@6a3af151 url=http://127.0.0.1:57573/imocl/r/collection1 node=127.0.0.1:57573_imocl%2Fr C273_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57573_imocl%2Fr, base_url=http://127.0.0.1:57573/imocl/r, leader=true}
[junit4:junit4] 2> 781607 T2676 C273 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 781618 T2680 C273 P57573 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 781619 T2680 C273 P57573 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 781620 T2680 C273 P57573 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 781620 T2680 C273 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 2
[junit4:junit4] 2> 781621 T2722 C272 P30511 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 781622 T2722 C272 P30511 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 781624 T2678 C273 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 781625 T2722 C272 P30511 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 781625 T2722 C272 P30511 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 781625 T2722 C272 P30511 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 781625 T2722 C272 P30511 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 781629 T2722 C272 P30511 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 781913 T2737 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 781914 T2737 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 781914 T2737 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 781928 T2737 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 781931 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 781932 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:30511_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:30511/imocl/r"}
[junit4:junit4] 2> 781932 T2737 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 781948 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 781948 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 781948 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 781948 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 781948 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 781948 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 781958 T2737 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 781962 T2737 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 781967 T2737 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 781968 T2737 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 781968 T2737 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 781968 T2737 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 781969 T2737 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 781969 T2737 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 781970 T2737 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 781970 T2737 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372427122551/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/
[junit4:junit4] 2> 781970 T2737 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@408fbecf
[junit4:junit4] 2> 781971 T2737 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 781972 T2737 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4
[junit4:junit4] 2> 781973 T2737 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index/
[junit4:junit4] 2> 781974 T2737 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 781975 T2737 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index
[junit4:junit4] 2> 781981 T2737 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 781981 T2737 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 781985 T2737 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 781985 T2737 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 781986 T2737 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 781986 T2737 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 781987 T2737 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 781987 T2737 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 781991 T2737 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 781992 T2737 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 781992 T2737 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 782002 T2737 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 782011 T2737 oass.SolrIndexSearcher.<init> Opening Searcher@40f00a98 main
[junit4:junit4] 2> 782012 T2737 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 782012 T2737 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 782023 T2739 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@40f00a98 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 782026 T2737 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 782026 T2737 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62504/imocl/r collection:collection1 shard:shard1
[junit4:junit4] 2> 782029 T2737 oasc.ZkController.register We are http://127.0.0.1:62504/imocl/r/collection1/ and leader is http://127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 782030 T2737 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62504/imocl/r
[junit4:junit4] 2> 782030 T2737 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 782030 T2737 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C274 name=collection1 org.apache.solr.core.SolrCore@34552052 url=http://127.0.0.1:62504/imocl/r/collection1 node=127.0.0.1:62504_imocl%2Fr C274_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:62504_imocl%2Fr, base_url=http://127.0.0.1:62504/imocl/r}
[junit4:junit4] 2> 782031 T2740 C274 P62504 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 782032 T2740 C274 P62504 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 782032 T2737 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 782032 T2740 C274 P62504 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 782032 T2740 C274 P62504 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 782033 T2643 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0
[junit4:junit4] 2> 782034 T2643 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 782034 T2740 C274 P62504 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 782034 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 782054 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 782054 T2681 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 782055 T2643 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 782055 T2643 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 782056 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 783058 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 783452 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 783453 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62504_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62504/imocl/r"}
[junit4:junit4] 2> 783473 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 783473 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 783473 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 783473 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 783473 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 783473 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 784057 T2681 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 784057 T2681 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:62504_imocl%252Fr&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 784060 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 785062 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C274_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:62504_imocl%2Fr, base_url=http://127.0.0.1:62504/imocl/r}
[junit4:junit4] 2> 786059 T2740 C274 P62504 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57573/imocl/r/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 786059 T2740 C274 P62504 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:62504/imocl/r START replicas=[http://127.0.0.1:57573/imocl/r/collection1/] nUpdates=100
[junit4:junit4] 2> 786060 T2740 C274 P62504 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 786060 T2740 C274 P62504 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 786060 T2740 C274 P62504 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 786060 T2740 C274 P62504 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 786060 T2740 C274 P62504 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 786061 T2740 C274 P62504 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57573/imocl/r/collection1/. core=collection1
[junit4:junit4] 2> 786061 T2740 C274 P62504 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C275 name=collection1 org.apache.solr.core.SolrCore@6a3af151 url=http://127.0.0.1:57573/imocl/r/collection1 node=127.0.0.1:57573_imocl%2Fr C275_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57573_imocl%2Fr, base_url=http://127.0.0.1:57573/imocl/r, leader=true}
[junit4:junit4] 2> 786062 T2676 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 786064 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 786071 T2675 C275 P57573 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 786072 T2675 C275 P57573 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 786073 T2675 C275 P57573 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 786073 T2675 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 2
[junit4:junit4] 2> 786074 T2740 C274 P62504 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 786074 T2740 C274 P62504 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 786077 T2678 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 786078 T2740 C274 P62504 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 786078 T2740 C274 P62504 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 786078 T2740 C274 P62504 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 786078 T2740 C274 P62504 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 786084 T2740 C274 P62504 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 786492 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 786493 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62504_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62504/imocl/r"}
[junit4:junit4] 2> 786560 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 786560 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 786560 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 786560 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 786560 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 786560 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 787066 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 787067 T2643 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C276 name=collection1 org.apache.solr.core.SolrCore@3207905b url=http://127.0.0.1:29692/imocl/r/collection1 node=127.0.0.1:29692_imocl%2Fr C276_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:29692_imocl%2Fr, base_url=http://127.0.0.1:29692/imocl/r, leader=true}
[junit4:junit4] 2> 787076 T2655 C276 P29692 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 787081 T2655 C276 P29692 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 787082 T2655 C276 P29692 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 787083 T2655 C276 P29692 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 787083 T2655 C276 P29692 oass.SolrIndexSearcher.<init> Opening Searcher@540948a7 main
[junit4:junit4] 2> 787084 T2655 C276 P29692 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 787085 T2669 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@540948a7 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 787085 T2655 C276 P29692 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> C274_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:62504_imocl%2Fr, base_url=http://127.0.0.1:62504/imocl/r}
[junit4:junit4] 2> 787104 T2726 C274 P62504 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:57573/imocl/r/collection1/, StdNode: http://127.0.0.1:40244/imocl/r/collection1/, StdNode: http://127.0.0.1:30511/imocl/r/collection1/, StdNode: http://127.0.0.1:62504/imocl/r/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 787107 T2677 C275 P57573 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 787107 T2727 C274 P62504 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 787108 T2677 C275 P57573 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 787109 T2677 C275 P57573 oass.SolrIndexSearcher.<init> Opening Searcher@37f8883a main
[junit4:junit4] 2> 787110 T2677 C275 P57573 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 787111 T2687 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@37f8883a main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 787111 T2727 C274 P62504 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 787111 T2677 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> 787112 T2727 C274 P62504 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 787113 T2727 C274 P62504 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 787114 T2727 C274 P62504 oass.SolrIndexSearcher.<init> Opening Searcher@275aa868 main
[junit4:junit4] 2> 787114 T2727 C274 P62504 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> ASYNC NEW_CORE C277 name=collection1 org.apache.solr.core.SolrCore@673e47ea url=http://127.0.0.1:30511/imocl/r/collection1 node=127.0.0.1:30511_imocl%2Fr C277_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:30511_imocl%2Fr, base_url=http://127.0.0.1:30511/imocl/r}
[junit4:junit4] 2> 787115 T2708 C277 P30511 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 787115 T2739 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@275aa868 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> ASYNC NEW_CORE C278 name=collection1 org.apache.solr.core.SolrCore@732620b url=http://127.0.0.1:40244/imocl/r/collection1 node=127.0.0.1:40244_imocl%2Fr C278_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:40244_imocl%2Fr, base_url=http://127.0.0.1:40244/imocl/r}
[junit4:junit4] 2> 787115 T2691 C278 P40244 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 787116 T2727 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 787119 T2708 C277 P30511 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 787120 T2708 C277 P30511 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 787120 T2708 C277 P30511 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 787121 T2691 C278 P40244 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 787121 T2708 C277 P30511 oass.SolrIndexSearcher.<init> Opening Searcher@68593f4e main
[junit4:junit4] 2> 787121 T2691 C278 P40244 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 787122 T2708 C277 P30511 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 787122 T2691 C278 P40244 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 787123 T2721 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68593f4e main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 787123 T2691 C278 P40244 oass.SolrIndexSearcher.<init> Opening Searcher@4d4a119a main
[junit4:junit4] 2> 787123 T2708 C277 P30511 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 787124 T2691 C278 P40244 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 787124 T2703 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d4a119a main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 787125 T2691 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 787126 T2726 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> 787127 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 787129 T2679 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 787131 T2692 C278 P40244 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 787133 T2709 C277 P30511 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 787135 T2728 C274 P62504 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 789140 T2655 C276 P29692 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439094154556604416)} 0 2
[junit4:junit4] 2> 789150 T2729 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={update.distrib=FROMLEADER&_version_=-1439094154563944448&update.from=http://127.0.0.1:57573/imocl/r/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439094154563944448)} 0 2
[junit4:junit4] 2> 789150 T2710 C277 P30511 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={update.distrib=FROMLEADER&_version_=-1439094154563944448&update.from=http://127.0.0.1:57573/imocl/r/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439094154563944448)} 0 1
[junit4:junit4] 2> 789150 T2694 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={update.distrib=FROMLEADER&_version_=-1439094154563944448&update.from=http://127.0.0.1:57573/imocl/r/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439094154563944448)} 0 2
[junit4:junit4] 2> 789152 T2680 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439094154563944448)} 0 7
[junit4:junit4] 2> 789152 T2693 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 10
[junit4:junit4] 2> 789158 T2655 C276 P29692 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={wt=javabin&version=2} {add=[0 (1439094154574430208)]} 0 3
[junit4:junit4] 2> 789171 T2694 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1439094154582818816)]} 0 3
[junit4:junit4] 2> 789172 T2710 C277 P30511 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1439094154582818816)]} 0 4
[junit4:junit4] 2> 789172 T2729 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1439094154582818816)]} 0 4
[junit4:junit4] 2> 789173 T2680 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:40244/imocl/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1439094154582818816)]} 0 10
[junit4:junit4] 2> 789173 T2693 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={wt=javabin&version=2} {add=[0]} 0 12
[junit4:junit4] 2> 789177 T2655 C276 P29692 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={wt=javabin&version=2} {add=[1 (1439094154596450304)]} 0 1
[junit4:junit4] 2> 789185 T2729 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439094154600644608)]} 0 1
[junit4:junit4] 2> 789185 T2710 C277 P30511 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439094154600644608)]} 0 1
[junit4:junit4] 2> 789185 T2694 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439094154600644608)]} 0 1
[junit4:junit4] 2> 789186 T2681 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:30511/imocl/r/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439094154600644608)]} 0 6
[junit4:junit4] 2> 789187 T2711 C277 P30511 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={wt=javabin&version=2} {add=[1]} 0 9
[junit4:junit4] 2> 789190 T2655 C276 P29692 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={wt=javabin&version=2} {add=[2 (1439094154610081792)]} 0 1
[junit4:junit4] 2> 789191 T2675 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:40244/imocl/r/collection1/]
[junit4:junit4] 2> 789192 T2675 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:40244/imocl/r/collection1/ against:[http://127.0.0.1:40244/imocl/r/collection1/] result:true
[junit4:junit4] 2> 789192 T2675 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:30511/imocl/r/collection1/ against:[http://127.0.0.1:40244/imocl/r/collection1/] result:false
[junit4:junit4] 2> 789193 T2675 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:62504/imocl/r/collection1/ against:[http://127.0.0.1:40244/imocl/r/collection1/] result:false
[junit4:junit4] 2> 789197 T2729 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:40244/imocl/r/collection1/&wt=javabin&version=2} {add=[2 (1439094154614276096)]} 0 1
[junit4:junit4] 2> 789197 T2710 C277 P30511 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:40244/imocl/r/collection1/&wt=javabin&version=2} {add=[2 (1439094154614276096)]} 0 1
[junit4:junit4] 2> 789198 T2675 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={test.distrib.skip.servers=http://127.0.0.1:40244/imocl/r/collection1/&wt=javabin&version=2} {add=[2 (1439094154614276096)]} 0 7
[junit4:junit4] 2> 789202 T2655 C276 P29692 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={wt=javabin&version=2} {add=[3 (1439094154622664704)]} 0 1
[junit4:junit4] 2> 789206 T2677 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:40244/imocl/r/collection1/]
[junit4:junit4] 2> 789206 T2677 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:40244/imocl/r/collection1/ against:[http://127.0.0.1:40244/imocl/r/collection1/] result:true
[junit4:junit4] 2> 789207 T2677 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:30511/imocl/r/collection1/ against:[http://127.0.0.1:40244/imocl/r/collection1/] result:false
[junit4:junit4] 2> 789207 T2677 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:62504/imocl/r/collection1/ against:[http://127.0.0.1:40244/imocl/r/collection1/] result:false
[junit4:junit4] 2> 789212 T2729 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:40244/imocl/r/collection1/&wt=javabin&version=2} {add=[3 (1439094154630004736)]} 0 1
[junit4:junit4] 2> 789212 T2710 C277 P30511 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:40244/imocl/r/collection1/&wt=javabin&version=2} {add=[3 (1439094154630004736)]} 0 1
[junit4:junit4] 2> 789213 T2677 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:62504/imocl/r/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:40244/imocl/r/collection1/&wt=javabin&version=2} {add=[3 (1439094154630004736)]} 0 7
[junit4:junit4] 2> 789213 T2726 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={test.distrib.skip.servers=http://127.0.0.1:40244/imocl/r/collection1/&test.distrib.skip.servers=http://127.0.0.1:30511/imocl/r/collection1/&wt=javabin&version=2} {add=[3]} 0 9
[junit4:junit4] 2> 789216 T2655 C276 P29692 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789264 T2655 C276 P29692 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 789265 T2655 C276 P29692 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 789271 T2655 C276 P29692 oass.SolrIndexSearcher.<init> Opening Searcher@4fe801f4 main
[junit4:junit4] 2> 789272 T2655 C276 P29692 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789273 T2669 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4fe801f4 main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 789274 T2655 C276 P29692 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 789277 T2693 C278 P40244 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:57573/imocl/r/collection1/, StdNode: http://127.0.0.1:40244/imocl/r/collection1/, StdNode: http://127.0.0.1:30511/imocl/r/collection1/, StdNode: http://127.0.0.1:62504/imocl/r/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 789279 T2695 C278 P40244 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789280 T2730 C274 P62504 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789279 T2712 C277 P30511 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789281 T2680 C275 P57573 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789303 T2695 C278 P40244 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 789304 T2695 C278 P40244 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 789305 T2730 C274 P62504 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 789305 T2730 C274 P62504 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 789306 T2712 C277 P30511 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty3/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 789306 T2680 C275 P57573 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 789307 T2712 C277 P30511 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 789307 T2680 C275 P57573 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 789312 T2695 C278 P40244 oass.SolrIndexSearcher.<init> Opening Searcher@1c8c37d0 main
[junit4:junit4] 2> 789313 T2695 C278 P40244 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789313 T2730 C274 P62504 oass.SolrIndexSearcher.<init> Opening Searcher@f374a22 main
[junit4:junit4] 2> 789314 T2703 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c8c37d0 main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C2)}
[junit4:junit4] 2> 789314 T2730 C274 P62504 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789315 T2712 C277 P30511 oass.SolrIndexSearcher.<init> Opening Searcher@7c6b3d66 main
[junit4:junit4] 2> 789315 T2739 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f374a22 main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 789315 T2695 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 36
[junit4:junit4] 2> 789315 T2680 C275 P57573 oass.SolrIndexSearcher.<init> Opening Searcher@4471c85c main
[junit4:junit4] 2> 789317 T2721 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7c6b3d66 main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 789317 T2680 C275 P57573 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789316 T2730 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 36
[junit4:junit4] 2> 789318 T2687 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4471c85c main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 789316 T2712 C277 P30511 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789318 T2680 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 37
[junit4:junit4] 2> 789318 T2712 C277 P30511 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 39
[junit4:junit4] 2> 789319 T2693 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 42
[junit4:junit4] 2> 789320 T2643 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 789321 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 789322 T2643 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 789324 T2679 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 789327 T2692 C278 P40244 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 789330 T2709 C277 P30511 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 789333 T2728 C274 P62504 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 789340 T2643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 789365 T2713 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 789366 T2713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 789379 T2755 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 789379 T2755 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 789379 T2755 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57573/imocl/r START replicas=[http://127.0.0.1:40244/imocl/r/collection1/, http://127.0.0.1:30511/imocl/r/collection1/, http://127.0.0.1:62504/imocl/r/collection1/] nUpdates=100
[junit4:junit4] 2> 789381 T2696 C278 P40244 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 789381 T2714 C277 P30511 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 789382 T2755 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:57573/imocl/r Received 3 versions from 127.0.0.1:40244/imocl/r/collection1/
[junit4:junit4] 2> 789382 T2731 C274 P62504 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 789382 T2755 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:57573/imocl/r Our versions are newer. ourLowThreshold=1439094154563944448 otherHigh=1439094154600644608
[junit4:junit4] 2> 789383 T2755 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:57573/imocl/r Received 5 versions from 127.0.0.1:30511/imocl/r/collection1/
[junit4:junit4] 2> 789384 T2755 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:57573/imocl/r Our versions are newer. ourLowThreshold=1439094154563944448 otherHigh=1439094154614276096
[junit4:junit4] 2> 789384 T2755 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:57573/imocl/r Received 5 versions from 127.0.0.1:62504/imocl/r/collection1/
[junit4:junit4] 2> 789384 T2755 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:57573/imocl/r Our versions are newer. ourLowThreshold=1439094154563944448 otherHigh=1439094154614276096
[junit4:junit4] 2> 789385 T2755 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57573/imocl/r DONE. sync succeeded
[junit4:junit4] 2> 789385 T2755 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 789385 T2755 oasc.SyncStrategy.syncToMe http://127.0.0.1:57573/imocl/r/collection1/: try and ask http://127.0.0.1:40244/imocl/r/collection1/ to sync
[junit4:junit4] 2> 789386 T2755 oasc.SyncStrategy.syncToMe http://127.0.0.1:57573/imocl/r/collection1/: try and ask http://127.0.0.1:30511/imocl/r/collection1/ to sync
[junit4:junit4] 2> 789386 T2755 oasc.SyncStrategy.syncToMe http://127.0.0.1:57573/imocl/r/collection1/: try and ask http://127.0.0.1:62504/imocl/r/collection1/ to sync
[junit4:junit4] 2> 789387 T2697 C278 P40244 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r START replicas=[http://127.0.0.1:57573/imocl/r/collection1/] nUpdates=100
[junit4:junit4] 2> 789388 T2762 C277 P30511 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:30511/imocl/r START replicas=[http://127.0.0.1:57573/imocl/r/collection1/] nUpdates=100
[junit4:junit4] 2> 789388 T2732 C274 P62504 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:62504/imocl/r START replicas=[http://127.0.0.1:57573/imocl/r/collection1/] nUpdates=100
[junit4:junit4] 2> 789389 T2766 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 789390 T2767 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 789390 T2762 C277 P30511 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:30511/imocl/r Received 5 versions from 127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 789390 T2676 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 789390 T2732 C274 P62504 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:62504/imocl/r Received 5 versions from 127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 789391 T2697 C278 P40244 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r Received 5 versions from 127.0.0.1:57573/imocl/r/collection1/
[junit4:junit4] 2> 789391 T2762 C277 P30511 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:30511/imocl/r Our versions are newer. ourLowThreshold=1439094154563944448 otherHigh=1439094154614276096
[junit4:junit4] 2> 789392 T2697 C278 P40244 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r Requesting updates from 127.0.0.1:57573/imocl/r/collection1/n=2 versions=[1439094154630004736, 1439094154614276096]
[junit4:junit4] 2> 789391 T2732 C274 P62504 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:62504/imocl/r Our versions are newer. ourLowThreshold=1439094154563944448 otherHigh=1439094154614276096
[junit4:junit4] 2> 789392 T2762 C277 P30511 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:30511/imocl/r DONE. sync succeeded
[junit4:junit4] 2> 789393 T2732 C274 P62504 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:62504/imocl/r DONE. sync succeeded
[junit4:junit4] 2> 789393 T2762 C277 P30511 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={sync=http://127.0.0.1:57573/imocl/r/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=5
[junit4:junit4] 2> 789394 T2732 C274 P62504 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={sync=http://127.0.0.1:57573/imocl/r/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=6
[junit4:junit4] 2> 789394 T2755 oasc.SyncStrategy.syncToMe http://127.0.0.1:57573/imocl/r/collection1/: sync completed with http://127.0.0.1:30511/imocl/r/collection1/
[junit4:junit4] 2> 789395 T2755 oasc.SyncStrategy.syncToMe http://127.0.0.1:57573/imocl/r/collection1/: sync completed with http://127.0.0.1:62504/imocl/r/collection1/
[junit4:junit4] 2> 789395 T2676 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={distrib=false&getUpdates=1439094154630004736,1439094154614276096&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 789401 T2697 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1439094154614276096), 3 (1439094154630004736)]} 0 5
[junit4:junit4] 2> 789402 T2697 C278 P40244 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r DONE. sync succeeded
[junit4:junit4] 2> 789402 T2697 C278 P40244 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={sync=http://127.0.0.1:57573/imocl/r/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=15
[junit4:junit4] 2> 789403 T2755 oasc.SyncStrategy.syncToMe http://127.0.0.1:57573/imocl/r/collection1/: sync completed with http://127.0.0.1:40244/imocl/r/collection1/
[junit4:junit4] 2> 789403 T2755 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=25
[junit4:junit4] 2> 789404 T2713 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=45
[junit4:junit4] 2> 789404 T2643 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 789405 T2643 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 789406 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 789407 T2643 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 789410 T2655 C276 P29692 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789411 T2655 C276 P29692 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 789412 T2655 C276 P29692 oass.SolrIndexSearcher.<init> Opening Searcher@737008ef main
[junit4:junit4] 2> 789413 T2655 C276 P29692 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789413 T2669 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@737008ef main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 789414 T2655 C276 P29692 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> 789417 T2675 C275 P57573 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:57573/imocl/r/collection1/, StdNode: http://127.0.0.1:40244/imocl/r/collection1/, StdNode: http://127.0.0.1:30511/imocl/r/collection1/, StdNode: http://127.0.0.1:62504/imocl/r/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 789419 T2769 C275 P57573 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789420 T2769 C275 P57573 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 789420 T2694 C278 P40244 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789420 T2769 C275 P57573 oass.SolrIndexSearcher.<init> Opening Searcher@1fa1118a main
[junit4:junit4] 2> 789420 T2729 C274 P62504 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789420 T2710 C277 P30511 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 789422 T2729 C274 P62504 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 789422 T2769 C275 P57573 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789424 T2687 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1fa1118a main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 789423 T2710 C277 P30511 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 789425 T2769 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 789425 T2710 C277 P30511 oass.SolrIndexSearcher.<init> Opening Searcher@2aaf13f1 main
[junit4:junit4] 2> 789425 T2729 C274 P62504 oass.SolrIndexSearcher.<init> Opening Searcher@56bf7ca6 main
[junit4:junit4] 2> 789426 T2710 C277 P30511 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789426 T2729 C274 P62504 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789427 T2721 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2aaf13f1 main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 789427 T2739 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@56bf7ca6 main{StandardDirectoryReader(segments_2:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 789427 T2710 C277 P30511 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 789427 T2729 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 789483 T2694 C278 P40244 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 789484 T2694 C278 P40244 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 789489 T2694 C278 P40244 oass.SolrIndexSearcher.<init> Opening Searcher@2c7f90c0 main
[junit4:junit4] 2> 789490 T2694 C278 P40244 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 789491 T2703 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c7f90c0 main{StandardDirectoryReader(segments_3:5:nrt _0(4.4):C2 _1(4.4):C2)}
[junit4:junit4] 2> 789491 T2694 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 71
[junit4:junit4] 2> 789492 T2675 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 75
[junit4:junit4] 2> 789493 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 789496 T2679 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 789499 T2692 C278 P40244 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 789502 T2709 C277 P30511 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 789505 T2728 C274 P62504 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 791507 T2643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> check const of shard1
[junit4:junit4] 2> client0
[junit4:junit4] 2> PROPS:2:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:57573_imocl%2Fr","base_url":"http://127.0.0.1:57573/imocl/r","leader":"true"}
[junit4:junit4] 2> 791512 T2679 C275 P57573 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client1
[junit4:junit4] 2> PROPS:3:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:40244_imocl%2Fr","base_url":"http://127.0.0.1:40244/imocl/r"}
[junit4:junit4] 2> 791514 T2692 C278 P40244 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client2
[junit4:junit4] 2> PROPS:4:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:30511_imocl%2Fr","base_url":"http://127.0.0.1:30511/imocl/r"}
[junit4:junit4] 2> 791517 T2709 C277 P30511 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client3
[junit4:junit4] 2> PROPS:5:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:62504_imocl%2Fr","base_url":"http://127.0.0.1:62504/imocl/r"}
[junit4:junit4] 2> 791520 T2728 C274 P62504 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> 791523 T2711 C277 P30511 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/select params={q=*:*&wt=javabin&version=2} hits=4 status=0 QTime=0
[junit4:junit4] 2> 791531 T2655 C276 P29692 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={wt=javabin&version=2} {add=[4 (1439094157061652480)]} 0 4
[junit4:junit4] 2> 791534 T2675 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:30511/imocl/r/collection1/]
[junit4:junit4] 2> 791535 T2675 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:40244/imocl/r/collection1/ against:[http://127.0.0.1:30511/imocl/r/collection1/] result:false
[junit4:junit4] 2> 791535 T2675 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:30511/imocl/r/collection1/ against:[http://127.0.0.1:30511/imocl/r/collection1/] result:true
[junit4:junit4] 2> 791535 T2675 C275 P57573 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:62504/imocl/r/collection1/ against:[http://127.0.0.1:30511/imocl/r/collection1/] result:false
[junit4:junit4] 2> 791548 T2729 C274 P62504 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:30511/imocl/r/collection1/&wt=javabin&version=2} {add=[4 (1439094157071089664)]} 0 4
[junit4:junit4] 2> 791548 T2694 C278 P40244 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={distrib.from=http://127.0.0.1:57573/imocl/r/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:30511/imocl/r/collection1/&wt=javabin&version=2} {add=[4 (1439094157071089664)]} 0 4
[junit4:junit4] 2> 791549 T2675 C275 P57573 oasup.LogUpdateProcessor.finish [collection1] webapp=/imocl/r path=/update params={test.distrib.skip.servers=http://127.0.0.1:30511/imocl/r/collection1/&wt=javabin&version=2} {add=[4 (1439094157071089664)]} 0 15
[junit4:junit4] 2> 791550 T2643 oasc.ChaosMonkey.monkeyLog monkey: kill shard! 57573
[junit4:junit4] 2> 791551 T2643 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=341553942
[junit4:junit4] 2> 792570 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 792571 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:57573_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57573/imocl/r"}
[junit4:junit4] 2> 792589 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 792589 T2701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 792589 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 792589 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 792589 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 792589 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 794575 T2643 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 794575 T2643 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 794577 T2643 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@6a3af151
[junit4:junit4] 2> 794589 T2643 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=6,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=1,adds=1,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=5,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 794589 T2643 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 794590 T2643 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 794590 T2643 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> C275_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57573_imocl%2Fr, base_url=http://127.0.0.1:57573/imocl/r, leader=true}
[junit4:junit4] 2> 794642 T2643 C275 P57573 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 794643 T2643 C275 P57573 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 794647 T2643 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 794647 T2643 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 794648 T2643 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1;done=false>>]
[junit4:junit4] 2> 794648 T2643 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1
[junit4:junit4] 2> 794649 T2643 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index;done=false>>]
[junit4:junit4] 2> 794649 T2643 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty1/index
[junit4:junit4] 2> 794651 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 794651 T2685 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 794652 T2643 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/imocl/r,null}
[junit4:junit4] 2> 794652 T2665 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 794651 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 794652 T2685 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 794653 T2672 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 794652 T2718 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 794653 T2685 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 794653 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 794654 T2701 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 794653 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 794654 T2736 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 794664 T2701 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 794664 T2701 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 794664 T2701 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 794664 T2701 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40244/imocl/r/collection1/
[junit4:junit4] 2> 794665 T2701 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r START replicas=[http://127.0.0.1:57573/imocl/r/collection1/, http://127.0.0.1:30511/imocl/r/collection1/, http://127.0.0.1:62504/imocl/r/collection1/] nUpdates=100
[junit4:junit4] 2> 794669 T2714 C277 P30511 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 794670 T2731 C274 P62504 oasc.SolrCore.execute [collection1] webapp=/imocl/r path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 794670 T2701 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r Received 5 versions from 127.0.0.1:30511/imocl/r/collection1/
[junit4:junit4] 2> 794671 T2701 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r Our versions are newer. ourLowThreshold=1439094154582818816 otherHigh=1439094154614276096
[junit4:junit4] 2> 794671 T2701 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r Received 6 versions from 127.0.0.1:62504/imocl/r/collection1/
[junit4:junit4] 2> 794671 T2701 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:40244/imocl/r Our versions are newer. ourLowThreshold=1439094154582818816 otherHigh=1439094154630004736
[junit4:junit4] 2> 794704 T2643 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 57573
[junit4:junit4] 2> 795595 T2666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 795596 T2666 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57573_imocl%2Fr",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57573/imocl/r"}
[junit4:junit4] 2> 795619 T2718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795619 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795619 T2672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795619 T2665 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 796705 T2643 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 90 for each attempt
[junit4:junit4] 2> 796705 T2643 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):90
[junit4:junit4] 2> 796706 T2643
[...truncated too long message...]
it4:junit4] 2> 1627484 T2736 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:62504/imocl/r DONE. sync succeeded
[junit4:junit4] 2> 1627485 T2736 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
[junit4:junit4] 2> 1627485 T2736 oasc.ShardLeaderElectionContext.runLeaderProcess Sync was not a success but no one else is active! I am the leader
[junit4:junit4] 2> 1627485 T2736 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:62504/imocl/r/collection1/
[junit4:junit4] 2> 1627485 T2736 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@34552052
[junit4:junit4] 2> 1627487 T2736 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=51,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=5,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 1627488 T2736 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 1627488 T2736 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1627488 T2736 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 1627491 T2736 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1627493 T2736 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1627494 T2736 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4;done=false>>]
[junit4:junit4] 2> 1627494 T2736 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4
[junit4:junit4] 2> 1627494 T2736 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index;done=false>>]
[junit4:junit4] 2> 1627494 T2736 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1372427105720/jetty4/index
[junit4:junit4] 2> 1627495 T2736 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1627495 T2736 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election/89943382801055756-5-n_0000000003
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:152)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:149)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:260)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 1627496 T2736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1627496 T2736 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1627496 T2736 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1627496 T2736 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene3x, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=it_CH, timezone=Africa/Banjul
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=3,free=176706904,total=439418880
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SynonymTokenizerTest, BinaryUpdateRequestHandlerTest, TestFunctionQuery, TestPluginEnable, ExternalFileFieldSortTest, UnloadDistributedZkTest, RecoveryZkTest, TestReversedWildcardFilterFactory, ScriptEngineTest, AutoCommitTest, BadComponentTest, StatelessScriptUpdateProcessorFactoryTest, SearchHandlerTest, TestDocumentBuilder, SolrTestCaseJ4Test, TestQueryUtils, TestXIncludeConfig, PluginInfoTest, TestIBSimilarityFactory, TestSolrQueryParser, TestDynamicFieldCollectionResource, UpdateRequestProcessorFactoryTest, TestDefaultSearchFieldResource, TestRemoteStreaming, PrimUtilsTest, SchemaVersionSpecificBehaviorTest, TestSolrQueryParserResource, OverseerTest, WordBreakSolrSpellCheckerTest, ResourceLoaderTest, TestLRUCache, ResponseLogComponentTest, LegacyHTMLStripCharFilterTest, TestWriterPerf, DirectUpdateHandlerOptimizeTest, CacheHeaderTest, JsonLoaderTest, ChaosMonkeyNothingIsSafeTest, SliceStateTest, TestSchemaNameResource, TestSolrQueryParserDefaultOperatorResource, TestPseudoReturnFields, PreAnalyzedUpdateProcessorTest, CSVRequestHandlerTest, TestReplicationHandler, TermVectorComponentTest, SolrCoreTest, MoreLikeThisHandlerTest, TestPhraseSuggestions, ShardRoutingTest, PeerSyncTest, SolrIndexSplitterTest, CircularListTest, TestDocSet, SOLR749Test, TestFieldTypeResource, SoftAutoCommitTest, TestFaceting, TestAddFieldRealTimeGet, TestFastWriter, TestFastLRUCache, SimpleFacetsTest, DistributedSpellCheckComponentTest, PrimitiveFieldTypeTest, TestCopyFieldCollectionResource, SyncSliceTest]
[junit4:junit4] Completed on J0 in 867.22s, 1 test, 1 failure <<< FAILURES!
[...truncated 477 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:392: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:372: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/common-build.xml:1248: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/common-build.xml:892: There were test failures: 299 suites, 1290 tests, 1 failure, 12 ignored (6 assumptions)
Total time: 57 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure