You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/01/25 16:51:07 UTC
[JENKINS] Lucene-Solr-Tests-4.x-Java6 - Build # 2282 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java6/2282/
1 tests failed.
REGRESSION: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:47768/collection2, http://127.0.0.1:17421/collection2, http://127.0.0.1:17415/collection2, http://127.0.0.1:49544/collection2, http://127.0.0.1:58728/collection2]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:47768/collection2, http://127.0.0.1:17421/collection2, http://127.0.0.1:17415/collection2, http://127.0.0.1:49544/collection2, http://127.0.0.1:58728/collection2]
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:352)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrServer(FullSolrCloudDistribCmdsTest.java:410)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:143)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:679)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: no servers hosting shard:
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:317)
... 46 more
Build Log:
[...truncated 9770 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> 409329 T816 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 409335 T816 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./solrtest-FullSolrCloudDistribCmdsTest-1390663135949
[junit4] 2> 409336 T816 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 409337 T817 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 409438 T816 oasc.ZkTestServer.run start zk server on port:35744
[junit4] 2> 409439 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 409444 T823 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4af2d44c name:ZooKeeperConnection Watcher:127.0.0.1:35744 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 409444 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 409444 T816 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 409458 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 409459 T825 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2043395d name:ZooKeeperConnection Watcher:127.0.0.1:35744/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 409460 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 409460 T816 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 409463 T816 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 409466 T816 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 409468 T816 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 409478 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 409479 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 409484 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 409484 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 409588 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 409589 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 409600 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 409600 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 409603 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 409604 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 409606 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 409607 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 409610 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 409610 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 409613 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 409613 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 409616 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 409616 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 409619 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 409620 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 409622 T816 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 409623 T816 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 409823 T816 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 409826 T816 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:58728
[junit4] 2> 409827 T816 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 409827 T816 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 409827 T816 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1390663136248
[junit4] 2> 409828 T816 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1390663136248/'
[junit4] 2> 409852 T816 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1390663136248/solr.xml
[junit4] 2> 409904 T816 oasc.CoreContainer.<init> New CoreContainer 1801592012
[junit4] 2> 409904 T816 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1390663136248/]
[junit4] 2> 409905 T816 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 409906 T816 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 409906 T816 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 409906 T816 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 409907 T816 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 409907 T816 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 409907 T816 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 409907 T816 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 409908 T816 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 409916 T816 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 409916 T816 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 409916 T816 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 409917 T816 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35744/solr
[junit4] 2> 409917 T816 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 409918 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 409920 T836 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7650cc27 name:ZooKeeperConnection Watcher:127.0.0.1:35744 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 409920 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 409923 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 409930 T838 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1506543e name:ZooKeeperConnection Watcher:127.0.0.1:35744/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 409931 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 409933 T816 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 409937 T816 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 409946 T816 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 409948 T816 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58728_
[junit4] 2> 409950 T816 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58728_
[junit4] 2> 409953 T816 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 409956 T816 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 409971 T816 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 409973 T816 oasc.Overseer.start Overseer (id=91138499278012419-127.0.0.1:58728_-n_0000000000) starting
[junit4] 2> 409986 T816 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 409990 T840 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 409991 T816 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 409991 T840 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 409999 T816 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 410001 T816 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 410004 T839 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 410010 T841 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 410010 T841 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 410011 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 410011 T841 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 410013 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 410013 T839 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58728",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58728_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 410014 T839 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 410014 T839 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 410023 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 410024 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 411013 T841 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 411013 T841 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1390663136248/collection1
[junit4] 2> 411013 T841 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 411014 T841 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 411015 T841 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 411016 T841 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1390663136248/collection1/'
[junit4] 2> 411018 T841 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1390663136248/collection1/lib/README' to classloader
[junit4] 2> 411018 T841 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1390663136248/collection1/lib/classes/' to classloader
[junit4] 2> 411081 T841 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 411151 T841 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 411253 T841 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 411265 T841 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 411825 T841 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 411829 T841 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 411832 T841 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 411837 T841 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 411864 T841 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 411864 T841 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1390663136248/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/control/data/
[junit4] 2> 411864 T841 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@307b4703
[junit4] 2> 411865 T841 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/control/data
[junit4] 2> 411866 T841 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/control/data/index/
[junit4] 2> 411866 T841 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 411866 T841 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/control/data/index
[junit4] 2> 411867 T841 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7e3d3158
[junit4] 2> 411868 T841 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@8e8ea0a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e8e9308),segFN=segments_1,generation=1}
[junit4] 2> 411868 T841 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 411872 T841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 411873 T841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 411873 T841 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 411873 T841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 411874 T841 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 411874 T841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 411875 T841 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 411875 T841 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 411875 T841 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 411876 T841 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 411876 T841 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 411877 T841 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 411877 T841 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 411878 T841 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 411878 T841 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 411879 T841 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 411888 T841 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 411894 T841 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 411895 T841 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 411895 T841 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=12.646484375, floorSegmentMB=1.80859375, forceMergeDeletesPctAllowed=4.803913234201957, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3266963668464724
[junit4] 2> 411896 T841 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@8e8ea0a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e8e9308),segFN=segments_1,generation=1}
[junit4] 2> 411897 T841 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 411897 T841 oass.SolrIndexSearcher.<init> Opening Searcher@3bf5b0e4 main
[junit4] 2> 411901 T842 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3bf5b0e4 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 411903 T841 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 411904 T816 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4] 2> 411904 T816 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 411904 T845 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58728 collection:control_collection shard:shard1
[junit4] 2> 411906 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 411906 T845 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 411908 T847 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@697f0acc name:ZooKeeperConnection Watcher:127.0.0.1:35744/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411908 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 411910 T816 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 411922 T816 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 411923 T845 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 411933 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 411934 T845 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 411934 T845 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C681 name=collection1 org.apache.solr.core.SolrCore@63c3227d url=http://127.0.0.1:58728/collection1 node=127.0.0.1:58728_ C681_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:58728, core=collection1, node_name=127.0.0.1:58728_}
[junit4] 2> 411934 T845 C681 P58728 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58728/collection1/
[junit4] 2> 411935 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 411935 T845 C681 P58728 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 411935 T845 C681 P58728 oasc.SyncStrategy.syncToMe http://127.0.0.1:58728/collection1/ has no replicas
[junit4] 2> 411936 T845 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58728/collection1/ shard1
[junit4] 2> 411936 T845 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 411938 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 411939 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 411939 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 411944 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 411947 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 412050 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 412050 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 412097 T845 oasc.ZkController.register We are http://127.0.0.1:58728/collection1/ and leader is http://127.0.0.1:58728/collection1/
[junit4] 2> 412097 T845 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58728
[junit4] 2> 412097 T845 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 412097 T845 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 412097 T845 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 412099 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 412099 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 412099 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 412099 T845 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 412100 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 412101 T839 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:58728",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58728_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 412110 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 412206 T816 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 412207 T816 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 412210 T816 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:15579
[junit4] 2> 412211 T816 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 412211 T816 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 412212 T816 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1390663138536
[junit4] 2> 412212 T816 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1390663138536/'
[junit4] 2> 412214 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 412214 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 412249 T816 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1390663138536/solr.xml
[junit4] 2> 412328 T816 oasc.CoreContainer.<init> New CoreContainer 1524119650
[junit4] 2> 412328 T816 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1390663138536/]
[junit4] 2> 412330 T816 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 412331 T816 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 412331 T816 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 412331 T816 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 412332 T816 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 412336 T816 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 412337 T816 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 412337 T816 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 412338 T816 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 412350 T816 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 412350 T816 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 412350 T816 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 412351 T816 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35744/solr
[junit4] 2> 412352 T816 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 412353 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412356 T858 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f1e793c name:ZooKeeperConnection Watcher:127.0.0.1:35744 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412356 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412359 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 412361 T860 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@732a71 name:ZooKeeperConnection Watcher:127.0.0.1:35744/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 412361 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 412371 T816 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 413375 T816 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:15579_
[junit4] 2> 413386 T816 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:15579_
[junit4] 2> 413389 T847 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 413389 T838 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 413389 T860 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 413398 T861 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 413398 T861 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 413400 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413400 T861 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 413400 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413401 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413409 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 413411 T839 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:15579",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:15579_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 413411 T839 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 413412 T839 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 413458 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 413580 T860 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 413580 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 413580 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 414401 T861 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 414401 T861 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1390663138536/collection1
[junit4] 2> 414401 T861 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 414402 T861 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 414402 T861 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 414403 T861 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1390663138536/collection1/'
[junit4] 2> 414405 T861 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1390663138536/collection1/lib/classes/' to classloader
[junit4] 2> 414406 T861 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1390663138536/collection1/lib/README' to classloader
[junit4] 2> 414468 T861 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 414535 T861 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 414637 T861 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 414649 T861 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 415201 T861 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 415205 T861 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 415208 T861 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 415213 T861 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 415240 T861 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 415240 T861 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1390663138536/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty1/
[junit4] 2> 415240 T861 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@307b4703
[junit4] 2> 415241 T861 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty1
[junit4] 2> 415242 T861 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty1/index/
[junit4] 2> 415242 T861 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 415242 T861 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty1/index
[junit4] 2> 415243 T861 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@557597b7
[junit4] 2> 415244 T861 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@d238868 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@f0a0101),segFN=segments_1,generation=1}
[junit4] 2> 415244 T861 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 415249 T861 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 415250 T861 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 415250 T861 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 415250 T861 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 415251 T861 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 415251 T861 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 415251 T861 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 415252 T861 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 415252 T861 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 415253 T861 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 415254 T861 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 415254 T861 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 415254 T861 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 415255 T861 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 415255 T861 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 415256 T861 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 415268 T861 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 415274 T861 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 415275 T861 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 415276 T861 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=12.646484375, floorSegmentMB=1.80859375, forceMergeDeletesPctAllowed=4.803913234201957, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3266963668464724
[junit4] 2> 415276 T861 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@d238868 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@f0a0101),segFN=segments_1,generation=1}
[junit4] 2> 415277 T861 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 415278 T861 oass.SolrIndexSearcher.<init> Opening Searcher@6120f51a main
[junit4] 2> 415282 T862 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6120f51a main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 415285 T861 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 415286 T816 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4] 2> 415287 T816 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 415286 T865 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:15579 collection:collection1 shard:shard1
[junit4] 2> 415288 T865 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 415309 T865 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 415311 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 415311 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 415311 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 415311 T865 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 415312 T865 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C682 name=collection1 org.apache.solr.core.SolrCore@7d27a2b6 url=http://127.0.0.1:15579/collection1 node=127.0.0.1:15579_ C682_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:15579, core=collection1, node_name=127.0.0.1:15579_}
[junit4] 2> 415312 T865 C682 P15579 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:15579/collection1/
[junit4] 2> 415312 T865 C682 P15579 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 415312 T865 C682 P15579 oasc.SyncStrategy.syncToMe http://127.0.0.1:15579/collection1/ has no replicas
[junit4] 2> 415313 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 415313 T865 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:15579/collection1/ shard1
[junit4] 2> 415313 T865 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 415316 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 415317 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 415317 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 415317 T860 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 415321 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 415324 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 415428 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 415428 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 415428 T860 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 415473 T865 oasc.ZkController.register We are http://127.0.0.1:15579/collection1/ and leader is http://127.0.0.1:15579/collection1/
[junit4] 2> 415473 T865 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:15579
[junit4] 2> 415473 T865 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 415473 T865 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 415473 T865 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 415475 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 415475 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 415475 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 415475 T865 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 415476 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 415477 T839 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:15579",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:15579_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 415486 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 415598 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 415598 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 415598 T860 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 415602 T816 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 415603 T816 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 415606 T816 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:47768
[junit4] 2> 415607 T816 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 415607 T816 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 415608 T816 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1390663141900
[junit4] 2> 415608 T816 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1390663141900/'
[junit4] 2> 415648 T816 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1390663141900/solr.xml
[junit4] 2> 415729 T816 oasc.CoreContainer.<init> New CoreContainer 2021637316
[junit4] 2> 415730 T816 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1390663141900/]
[junit4] 2> 415732 T816 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 415732 T816 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 415733 T816 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 415733 T816 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 415733 T816 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 415734 T816 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 415734 T816 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 415735 T816 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 415735 T816 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 415747 T816 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 415748 T816 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 415749 T816 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 415750 T816 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35744/solr
[junit4] 2> 415750 T816 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 415752 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 415755 T876 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a4225aa name:ZooKeeperConnection Watcher:127.0.0.1:35744 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 415755 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 415758 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 415760 T878 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b010a3b name:ZooKeeperConnection Watcher:127.0.0.1:35744/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 415760 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 415772 T816 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 416776 T816 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47768_
[junit4] 2> 416778 T816 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47768_
[junit4] 2> 416781 T847 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 416781 T878 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 416781 T838 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 416781 T860 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 416795 T879 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 416795 T879 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 416796 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 416796 T879 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 416796 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 416797 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 416798 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 416799 T839 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:47768",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47768_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 416799 T839 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 416799 T839 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 416802 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 416904 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 416904 T878 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 416904 T860 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 416904 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 417798 T879 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 417798 T879 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1390663141900/collection1
[junit4] 2> 417798 T879 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 417799 T879 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 417799 T879 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 417800 T879 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1390663141900/collection1/'
[junit4] 2> 417802 T879 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1390663141900/collection1/lib/README' to classloader
[junit4] 2> 417803 T879 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1390663141900/collection1/lib/classes/' to classloader
[junit4] 2> 417865 T879 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 417937 T879 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 418038 T879 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 418049 T879 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 418604 T879 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 418608 T879 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 418610 T879 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 418616 T879 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 418646 T879 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 418647 T879 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1390663141900/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty2/
[junit4] 2> 418647 T879 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@307b4703
[junit4] 2> 418648 T879 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty2
[junit4] 2> 418648 T879 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty2/index/
[junit4] 2> 418649 T879 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 418649 T879 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty2/index
[junit4] 2> 418650 T879 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7ce0c4a3
[junit4] 2> 418651 T879 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1aae98b6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6adcf674),segFN=segments_1,generation=1}
[junit4] 2> 418651 T879 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 418656 T879 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 418657 T879 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 418657 T879 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 418657 T879 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 418658 T879 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 418658 T879 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 418658 T879 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 418659 T879 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 418659 T879 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 418660 T879 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 418660 T879 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 418661 T879 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 418661 T879 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 418662 T879 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 418662 T879 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 418663 T879 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 418673 T879 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 418686 T879 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 418686 T879 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 418687 T879 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=12.646484375, floorSegmentMB=1.80859375, forceMergeDeletesPctAllowed=4.803913234201957, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3266963668464724
[junit4] 2> 418688 T879 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1aae98b6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6adcf674),segFN=segments_1,generation=1}
[junit4] 2> 418688 T879 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 418689 T879 oass.SolrIndexSearcher.<init> Opening Searcher@4e426b0b main
[junit4] 2> 418694 T880 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e426b0b main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 418697 T879 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 418698 T816 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4] 2> 418698 T816 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 418698 T883 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47768 collection:collection1 shard:shard2
[junit4] 2> 418700 T883 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 418725 T883 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 418727 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 418727 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 418727 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 418727 T883 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 418728 T883 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C683 name=collection1 org.apache.solr.core.SolrCore@31b74ad6 url=http://127.0.0.1:47768/collection1 node=127.0.0.1:47768_ C683_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:47768, core=collection1, node_name=127.0.0.1:47768_}
[junit4] 2> 418728 T883 C683 P47768 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47768/collection1/
[junit4] 2> 418728 T883 C683 P47768 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 418729 T883 C683 P47768 oasc.SyncStrategy.syncToMe http://127.0.0.1:47768/collection1/ has no replicas
[junit4] 2> 418729 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 418729 T883 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47768/collection1/ shard2
[junit4] 2> 418730 T883 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 418733 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 418739 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 418739 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 418739 T878 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 418739 T860 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 418750 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 418759 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 418863 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 418863 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 418863 T878 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 418863 T860 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 418903 T883 oasc.ZkController.register We are http://127.0.0.1:47768/collection1/ and leader is http://127.0.0.1:47768/collection1/
[junit4] 2> 418903 T883 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47768
[junit4] 2> 418903 T883 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 418903 T883 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 418903 T883 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 418905 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 418905 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 418905 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 418905 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 418906 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 418907 T839 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:47768",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47768_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 418910 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 418992 T816 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 418993 T816 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 418995 T816 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:49544
[junit4] 2> 418996 T816 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 418997 T816 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 418997 T816 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1390663145312
[junit4] 2> 418997 T816 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1390663145312/'
[junit4] 2> 419014 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 419014 T878 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 419014 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 419014 T860 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 419035 T816 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1390663145312/solr.xml
[junit4] 2> 419113 T816 oasc.CoreContainer.<init> New CoreContainer 722692893
[junit4] 2> 419113 T816 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1390663145312/]
[junit4] 2> 419115 T816 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 419116 T816 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 419116 T816 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 419117 T816 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 419117 T816 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 419117 T816 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 419118 T816 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 419118 T816 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 419119 T816 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 419131 T816 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 419131 T816 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 419132 T816 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 419132 T816 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35744/solr
[junit4] 2> 419133 T816 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 419134 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 419167 T894 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7dd5303 name:ZooKeeperConnection Watcher:127.0.0.1:35744 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 419168 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 419172 T816 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 419185 T896 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@127029d0 name:ZooKeeperConnection Watcher:127.0.0.1:35744/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 419185 T816 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 419190 T816 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 420193 T816 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49544_
[junit4] 2> 420220 T816 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49544_
[junit4] 2> 420224 T847 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 420224 T896 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 420224 T878 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 420224 T860 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 420224 T838 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 420233 T897 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 420234 T897 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 420235 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 420235 T897 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 420236 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 420236 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 420238 T839 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 420239 T839 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49544",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49544_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 420239 T839 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 420239 T839 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 420253 T838 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 420358 T847 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 420358 T838 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 420358 T896 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 420358 T878 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 420358 T860 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 421237 T897 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 421237 T897 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1390663145312/collection1
[junit4] 2> 421237 T897 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 421238 T897 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 421238 T897 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 421240 T897 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1390663145312/collection1/'
[junit4] 2> 421241 T897 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1390663145312/collection1/lib/classes/' to classloader
[junit4] 2> 421241 T897 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1390663145312/collection1/lib/README' to classloader
[junit4] 2> 421311 T897 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 421437 T897 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 421539 T897 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 421566 T897 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 422974 T897 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 422984 T897 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 422988 T897 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 423000 T897 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 423054 T897 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 423055 T897 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1390663145312/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1390663135948/jetty3/
[junit4] 2> 423055 T897 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@307b4703
[junit4] 2> 423056 T897 oasc.CachingDirectoryFactory.get return new directory for
[...truncated too long message...]
0 T933 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty5-1390663153515/collection2_shard7_replica2/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty5-1390663153515/collection2_shard7_replica2/data;done=false>>]
[junit4] 2> 501080 T933 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty5-1390663153515/collection2_shard7_replica2/data
[junit4] 2> 501081 T933 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty5-1390663153515/collection2_shard7_replica2/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty5-1390663153515/collection2_shard7_replica2/data/index;done=false>>]
[junit4] 2> 501081 T933 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty5-1390663153515/collection2_shard7_replica2/data/index
[junit4] 2> 501081 T933 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection2/leader_elect/shard12/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 501082 T933 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection2/leader_elect/shard29/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 501082 T933 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 501083 T933 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 501083 T933 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection2/leader_elect/shard19/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 501083 T933 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection2/leader_elect/shard14/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Lucene45, sim=DefaultSimilarity, locale=en_PH, timezone=America/Winnipeg
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=1,free=235707608,total=458883072
[junit4] 2> NOTE: All tests run in this JVM: [BasicZkTest, TestDFRSimilarityFactory, TestFieldCollectionResource, OpenCloseCoreStressTest, AliasIntegrationTest, ChaosMonkeySafeLeaderTest, TestCSVLoader, IndexSchemaRuntimeFieldTest, FullSolrCloudDistribCmdsTest]
[junit4] Completed on J1 in 92.36s, 1 test, 1 error <<< FAILURES!
[...truncated 892 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:459: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:439: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/common-build.xml:491: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/common-build.xml:1308: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/common-build.xml:941: There were test failures: 355 suites, 1547 tests, 1 error, 35 ignored (7 assumptions)
Total time: 64 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure