You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/07/03 18:18:45 UTC
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1687
- Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1687/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
shard3 is not consistent. Got 145 from http://127.0.0.1:49730/collection1lastClient and got 146 from http://127.0.0.1:49740/collection1
Stack Trace:
java.lang.AssertionError: shard3 is not consistent. Got 145 from http://127.0.0.1:49730/collection1lastClient and got 146 from http://127.0.0.1:49740/collection1
at __randomizedtesting.SeedInfo.seed([BB315BB314BD9917:3AD7D5AB63E2F92B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1132)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1111)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 10846 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/init-core-data-001
[junit4] 2> 600120 T1802 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 600120 T1802 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 600127 T1802 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 600129 T1802 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 600130 T1803 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 600231 T1802 oasc.ZkTestServer.run start zk server on port:49720
[junit4] 2> 600237 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 600247 T1809 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34bcbc5b name:ZooKeeperConnection Watcher:127.0.0.1:49720 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 600248 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 600248 T1802 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 600306 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 600317 T1811 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39353f5d name:ZooKeeperConnection Watcher:127.0.0.1:49720/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 600318 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 600318 T1802 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 600330 T1802 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 600337 T1802 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 600342 T1802 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 600350 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 600350 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 600359 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 600360 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 600368 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 600369 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 600375 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 600376 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 600384 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 600385 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 600391 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 600392 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 600398 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 600399 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 600406 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 600407 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 600413 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 600414 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 600420 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 600422 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 600427 T1802 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 600428 T1802 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 602189 T1802 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 602199 T1802 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49723
[junit4] 2> 602200 T1802 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 602200 T1802 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 602200 T1802 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-002
[junit4] 2> 602200 T1802 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-002/'
[junit4] 2> 602239 T1802 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-002/solr.xml
[junit4] 2> 602264 T1802 oasc.CoreContainer.<init> New CoreContainer 90558478
[junit4] 2> 602264 T1802 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-002/]
[junit4] 2> 602265 T1802 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 602266 T1802 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 602266 T1802 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 602266 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 602266 T1802 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 602266 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 602268 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 602268 T1802 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 602268 T1802 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 602269 T1802 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 602269 T1802 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 602269 T1802 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 602270 T1802 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49720/solr
[junit4] 2> 602270 T1802 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 602271 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 602276 T1822 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2720de4 name:ZooKeeperConnection Watcher:127.0.0.1:49720 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 602277 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 602282 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 602286 T1824 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e715e7b name:ZooKeeperConnection Watcher:127.0.0.1:49720/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 602286 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 602290 T1802 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 602299 T1802 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 602307 T1802 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 602315 T1802 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 602323 T1802 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 602331 T1802 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 602334 T1802 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49723_
[junit4] 2> 602336 T1802 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49723_
[junit4] 2> 602344 T1802 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 602349 T1802 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 602359 T1802 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:49723_
[junit4] 2> 602359 T1802 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 602365 T1802 oasc.Overseer.start Overseer (id=92038820339580931-127.0.0.1:49723_-n_0000000000) starting
[junit4] 2> 602372 T1802 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 602408 T1826 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 602410 T1802 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 602418 T1802 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 602423 T1802 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 602430 T1825 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 602435 T1827 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 602435 T1827 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 602438 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 602438 T1827 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 602442 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 602443 T1825 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49723",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49723_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 602443 T1825 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 602443 T1825 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 602449 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 602452 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 603440 T1827 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 603440 T1827 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 603442 T1827 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 603442 T1827 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 603448 T1827 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 603448 T1827 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-002/collection1/'
[junit4] 2> 603450 T1827 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 603450 T1827 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 603451 T1827 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 603508 T1827 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 603538 T1827 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 603542 T1827 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 603546 T1827 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 603759 T1827 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 603762 T1827 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 603763 T1827 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 603772 T1827 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 603843 T1827 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 603844 T1827 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 603845 T1827 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-002/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/control/data/
[junit4] 2> 603845 T1827 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@50ecb538
[junit4] 2> 603846 T1827 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/control/data
[junit4] 2> 603847 T1827 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/control/data/index/
[junit4] 2> 603848 T1827 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 603849 T1827 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/control/data/index
[junit4] 2> 603849 T1827 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=63.982421875, floorSegmentMB=1.373046875, forceMergeDeletesPctAllowed=4.106268888577728, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 603855 T1827 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 603856 T1827 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 603865 T1827 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 603866 T1827 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 603866 T1827 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 603867 T1827 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 603867 T1827 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 603867 T1827 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 603868 T1827 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 603869 T1827 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 603869 T1827 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 603870 T1827 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 603870 T1827 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 603870 T1827 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 603871 T1827 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 603872 T1827 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 603872 T1827 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 603873 T1827 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 603874 T1827 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 603884 T1827 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 603886 T1827 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 603886 T1827 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 603887 T1827 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=48.423828125, floorSegmentMB=1.2548828125, forceMergeDeletesPctAllowed=2.3333866392119393, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 603889 T1827 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 603890 T1827 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 603890 T1827 oass.SolrIndexSearcher.<init> Opening Searcher@56ee94ab[collection1] main
[junit4] 2> 603890 T1827 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 603894 T1827 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 603894 T1827 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 603896 T1827 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 603896 T1827 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 603898 T1827 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 603899 T1827 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 603899 T1827 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 603901 T1827 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 603908 T1827 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 603908 T1827 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 603908 T1827 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 603911 T1828 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@56ee94ab[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 603912 T1827 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 603914 T1831 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49723 collection:control_collection shard:shard1
[junit4] 2> 603915 T1802 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 603915 T1802 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 603916 T1831 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 603918 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 603922 T1833 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48ea3576 name:ZooKeeperConnection Watcher:127.0.0.1:49720/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 603923 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 603927 T1802 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 603939 T1802 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 603940 T1831 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 603945 T1831 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 603946 T1831 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3066 name=collection1 org.apache.solr.core.SolrCore@137601fe url=http://127.0.0.1:49723/collection1 node=127.0.0.1:49723_ C3066_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:49723, core=collection1, node_name=127.0.0.1:49723_}
[junit4] 2> 603946 T1831 C3066 P49723 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49723/collection1/
[junit4] 2> 603946 T1831 C3066 P49723 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 603947 T1831 C3066 P49723 oasc.SyncStrategy.syncToMe http://127.0.0.1:49723/collection1/ has no replicas
[junit4] 2> 603947 T1831 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49723/collection1/ shard1
[junit4] 2> 603947 T1831 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 603949 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 603955 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 603964 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 603967 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 603969 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 603982 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 603990 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 604096 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 604097 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 604133 T1831 oasc.ZkController.register We are http://127.0.0.1:49723/collection1/ and leader is http://127.0.0.1:49723/collection1/
[junit4] 2> 604134 T1831 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49723
[junit4] 2> 604134 T1831 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 604134 T1831 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 604134 T1831 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 604139 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 604139 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 604140 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 604141 T1831 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 604146 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 604148 T1825 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:49723",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49723_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 604155 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 604264 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 604266 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 605660 T1802 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 605661 T1802 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 605677 T1802 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49727
[junit4] 2> 605677 T1802 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 605678 T1802 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 605679 T1802 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-003
[junit4] 2> 605679 T1802 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-003/'
[junit4] 2> 605720 T1802 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-003/solr.xml
[junit4] 2> 605747 T1802 oasc.CoreContainer.<init> New CoreContainer 924649623
[junit4] 2> 605747 T1802 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-003/]
[junit4] 2> 605748 T1802 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 605748 T1802 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 605748 T1802 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 605749 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 605749 T1802 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 605749 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 605749 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 605750 T1802 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 605750 T1802 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 605751 T1802 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 605752 T1802 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 605752 T1802 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 605753 T1802 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49720/solr
[junit4] 2> 605753 T1802 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 605755 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 605759 T1844 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27b085a3 name:ZooKeeperConnection Watcher:127.0.0.1:49720 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 605760 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 605768 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 605772 T1846 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5338ae98 name:ZooKeeperConnection Watcher:127.0.0.1:49720/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 605773 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 605798 T1802 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 606810 T1802 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49727_
[junit4] 2> 606814 T1802 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49727_
[junit4] 2> 606841 T1847 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 606842 T1847 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 606845 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 606846 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 606846 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 606847 T1847 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 606850 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 606851 T1825 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49727",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49727_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 606852 T1825 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 606852 T1825 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 606859 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 606860 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 606860 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 606861 T1846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 607848 T1847 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 607849 T1847 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 607857 T1847 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 607857 T1847 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 607860 T1847 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 607860 T1847 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-003/collection1/'
[junit4] 2> 607861 T1847 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 607862 T1847 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 607862 T1847 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 607913 T1847 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 607939 T1847 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 607942 T1847 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 607950 T1847 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 608187 T1847 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 608187 T1847 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 608189 T1847 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 608194 T1847 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 608267 T1847 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 608267 T1847 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 608267 T1847 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-003/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty1/
[junit4] 2> 608268 T1847 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@50ecb538
[junit4] 2> 608269 T1847 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty1
[junit4] 2> 608270 T1847 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty1/index/
[junit4] 2> 608271 T1847 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 608273 T1847 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty1/index
[junit4] 2> 608273 T1847 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=63.982421875, floorSegmentMB=1.373046875, forceMergeDeletesPctAllowed=4.106268888577728, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 608277 T1847 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 608277 T1847 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 608283 T1847 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 608284 T1847 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 608284 T1847 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 608285 T1847 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 608286 T1847 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 608287 T1847 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 608288 T1847 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 608288 T1847 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 608288 T1847 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 608289 T1847 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 608289 T1847 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 608290 T1847 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 608290 T1847 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 608291 T1847 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 608291 T1847 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 608292 T1847 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 608292 T1847 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 608301 T1847 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 608304 T1847 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 608304 T1847 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 608305 T1847 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=48.423828125, floorSegmentMB=1.2548828125, forceMergeDeletesPctAllowed=2.3333866392119393, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 608308 T1847 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 608309 T1847 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 608309 T1847 oass.SolrIndexSearcher.<init> Opening Searcher@438e8752[collection1] main
[junit4] 2> 608309 T1847 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 608312 T1847 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 608313 T1847 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 608314 T1847 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 608315 T1847 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 608315 T1847 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 608318 T1847 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 608318 T1847 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 608318 T1847 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 608318 T1847 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 608326 T1848 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@438e8752[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 608328 T1847 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 608330 T1851 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49727 collection:collection1 shard:shard2
[junit4] 2> 608331 T1802 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 608331 T1802 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 608332 T1851 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 608348 T1851 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 608355 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 608359 T1851 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 608359 T1851 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3067 name=collection1 org.apache.solr.core.SolrCore@1bd2d6b5 url=http://127.0.0.1:49727/collection1 node=127.0.0.1:49727_ C3067_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:49727, core=collection1, node_name=127.0.0.1:49727_}
[junit4] 2> 608359 T1851 C3067 P49727 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49727/collection1/
[junit4] 2> 608360 T1851 C3067 P49727 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 608360 T1851 C3067 P49727 oasc.SyncStrategy.syncToMe http://127.0.0.1:49727/collection1/ has no replicas
[junit4] 2> 608360 T1851 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49727/collection1/ shard2
[junit4] 2> 608360 T1851 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 608365 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 608378 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 608380 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 608381 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 608382 T1846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 608396 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 608405 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 608511 T1846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 608512 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 608512 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 608548 T1851 oasc.ZkController.register We are http://127.0.0.1:49727/collection1/ and leader is http://127.0.0.1:49727/collection1/
[junit4] 2> 608548 T1851 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49727
[junit4] 2> 608548 T1851 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 608549 T1851 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 608549 T1851 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 608561 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 608561 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 608561 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 608566 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 608566 T1851 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 608568 T1825 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:49727",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49727_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 608576 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 608683 T1846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 608684 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 608684 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 610052 T1802 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 610053 T1802 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 610056 T1802 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49730
[junit4] 2> 610058 T1802 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 610058 T1802 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 610058 T1802 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-004
[junit4] 2> 610059 T1802 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-004/'
[junit4] 2> 610094 T1802 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-004/solr.xml
[junit4] 2> 610124 T1802 oasc.CoreContainer.<init> New CoreContainer 1615614150
[junit4] 2> 610125 T1802 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-004/]
[junit4] 2> 610126 T1802 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 610126 T1802 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 610126 T1802 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 610126 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 610127 T1802 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 610127 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 610127 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 610127 T1802 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 610128 T1802 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 610128 T1802 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 610129 T1802 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 610129 T1802 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 610129 T1802 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49720/solr
[junit4] 2> 610130 T1802 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 610131 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 610137 T1862 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2449650a name:ZooKeeperConnection Watcher:127.0.0.1:49720 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 610137 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 610144 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 610148 T1864 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fece9c8 name:ZooKeeperConnection Watcher:127.0.0.1:49720/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 610148 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 610162 T1802 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 611169 T1802 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49730_
[junit4] 2> 611172 T1802 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49730_
[junit4] 2> 611196 T1865 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 611196 T1865 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 611199 T1865 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 611200 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 611200 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 611200 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 611203 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 611204 T1825 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49730",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49730_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 611205 T1825 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 611205 T1825 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 611210 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 611212 T1846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 611212 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 611213 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 611213 T1864 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 612200 T1865 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 612201 T1865 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 612203 T1865 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 612204 T1865 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 612208 T1865 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 612208 T1865 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-004/collection1/'
[junit4] 2> 612210 T1865 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 612211 T1865 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 612212 T1865 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 612303 T1865 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 612329 T1865 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 612342 T1865 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 612354 T1865 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 612553 T1865 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 612555 T1865 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 612556 T1865 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 612562 T1865 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 612634 T1865 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 612634 T1865 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 612635 T1865 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-004/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty2/
[junit4] 2> 612635 T1865 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@50ecb538
[junit4] 2> 612636 T1865 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty2
[junit4] 2> 612638 T1865 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty2/index/
[junit4] 2> 612639 T1865 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 612641 T1865 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty2/index
[junit4] 2> 612642 T1865 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=63.982421875, floorSegmentMB=1.373046875, forceMergeDeletesPctAllowed=4.106268888577728, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 612646 T1865 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 612647 T1865 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 612653 T1865 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 612655 T1865 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 612655 T1865 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 612655 T1865 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 612656 T1865 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 612656 T1865 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 612662 T1865 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 612662 T1865 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 612663 T1865 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 612663 T1865 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 612664 T1865 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 612664 T1865 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 612664 T1865 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 612665 T1865 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 612665 T1865 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 612666 T1865 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 612666 T1865 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 612674 T1865 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 612676 T1865 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 612677 T1865 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 612678 T1865 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=48.423828125, floorSegmentMB=1.2548828125, forceMergeDeletesPctAllowed=2.3333866392119393, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 612680 T1865 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 612681 T1865 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 612681 T1865 oass.SolrIndexSearcher.<init> Opening Searcher@6b6ae8f9[collection1] main
[junit4] 2> 612681 T1865 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 612689 T1865 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 612689 T1865 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 612691 T1865 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 612691 T1865 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 612691 T1865 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 612695 T1865 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 612695 T1865 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 612695 T1865 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 612696 T1865 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 612701 T1866 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b6ae8f9[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 612704 T1865 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 612708 T1869 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49730 collection:collection1 shard:shard3
[junit4] 2> 612709 T1802 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 612709 T1802 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 612711 T1869 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 612730 T1869 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
[junit4] 2> 612735 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 612737 T1869 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 612737 T1869 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3068 name=collection1 org.apache.solr.core.SolrCore@76adb157 url=http://127.0.0.1:49730/collection1 node=127.0.0.1:49730_ C3068_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:49730, core=collection1, node_name=127.0.0.1:49730_}
[junit4] 2> 612738 T1869 C3068 P49730 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49730/collection1/
[junit4] 2> 612738 T1869 C3068 P49730 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 612738 T1869 C3068 P49730 oasc.SyncStrategy.syncToMe http://127.0.0.1:49730/collection1/ has no replicas
[junit4] 2> 612738 T1869 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49730/collection1/ shard3
[junit4] 2> 612739 T1869 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 612742 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 612752 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 612755 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 612756 T1846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 612757 T1864 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 612758 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 612774 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 612783 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 612889 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 612889 T1864 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 612890 T1846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 612890 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 612923 T1869 oasc.ZkController.register We are http://127.0.0.1:49730/collection1/ and leader is http://127.0.0.1:49730/collection1/
[junit4] 2> 612924 T1869 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49730
[junit4] 2> 612924 T1869 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 612924 T1869 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 612924 T1869 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 612927 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 612927 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 612928 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 612929 T1869 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 612933 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 612935 T1825 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:49730",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49730_",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 612941 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 613047 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 613048 T1864 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 613049 T1846 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 613049 T1824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 614640 T1802 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 614640 T1802 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 614646 T1802 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49733
[junit4] 2> 614647 T1802 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 614647 T1802 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 614648 T1802 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-005
[junit4] 2> 614649 T1802 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-005/'
[junit4] 2> 614688 T1802 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-005/solr.xml
[junit4] 2> 614721 T1802 oasc.CoreContainer.<init> New CoreContainer 1836716321
[junit4] 2> 614722 T1802 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-005/]
[junit4] 2> 614723 T1802 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 614723 T1802 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 614723 T1802 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 614724 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 614724 T1802 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 614725 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 614726 T1802 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 614726 T1802 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 614727 T1802 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 614728 T1802 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 614728 T1802 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 614728 T1802 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 614729 T1802 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49720/solr
[junit4] 2> 614729 T1802 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 614731 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 614736 T1880 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b815b3b name:ZooKeeperConnection Watcher:127.0.0.1:49720 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 614738 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 614746 T1802 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 614752 T1882 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6aa571f6 name:ZooKeeperConnection Watcher:127.0.0.1:49720/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 614752 T1802 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 614774 T1802 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 615783 T1802 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49733_
[junit4] 2> 615786 T1802 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49733_
[junit4] 2> 615813 T1883 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 615813 T1883 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 615819 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 615819 T1883 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 615819 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 615821 T1824 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 615825 T1825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 615827 T1825 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49733",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49
[...truncated too long message...]
lr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty12/index.20140703222325477;done=false>>]
[junit4] 2> 725397 T2067 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001/tempDir-001/jetty12/index.20140703222325477
[junit4] 2> 725397 T2067 oasc.LeaderElector$ElectionWatcher.process WARN java.lang.RuntimeException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:1301)
[junit4] 2> at org.apache.solr.cloud.Overseer.getInQueue(Overseer.java:1252)
[junit4] 2> at org.apache.solr.cloud.Overseer.getInQueue(Overseer.java:1248)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:198)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:452)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:216)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:312)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:309)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:309)
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:1292)
[junit4] 2> ... 15 more
[junit4] 2>
[junit4] 2> 725398 T2067 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 725398 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725398 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725399 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725399 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725399 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725399 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725399 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725399 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725399 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725400 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725400 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725400 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725400 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725400 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725401 T2067 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4] 2> 725401 T2067 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 725401 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725401 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 725402 T2067 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-BB315BB314BD9917-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_UY, timezone=Asia/Bangkok
[junit4] 2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_05 (64-bit)/cpus=2,threads=1,free=131803600,total=226664448
[junit4] 2> NOTE: All tests run in this JVM: [TestHighlightDedupGrouping, TestSolrQueryParserDefaultOperatorResource, TestFieldTypeResource, TestRangeQuery, UpdateParamsTest, TestDFRSimilarityFactory, DateFieldTest, QueryElevationComponentTest, RAMDirectoryFactoryTest, TestReload, HdfsBasicDistributedZk2Test, TestUpdate, IndexSchemaRuntimeFieldTest, PluginInfoTest, TestFastOutputStream, TestSolrQueryParser, TestSchemaVersionResource, RecoveryZkTest, MultiTermTest, StatelessScriptUpdateProcessorFactoryTest, AtomicUpdatesTest, StatsComponentTest, DistribCursorPagingTest, DocValuesMissingTest, TestSolrDeletionPolicy1, TestStressLucene, IndexSchemaTest, TestRandomDVFaceting, PreAnalyzedFieldTest, TestReversedWildcardFilterFactory, CoreAdminCreateDiscoverTest, TestStressVersions, DistribDocExpirationUpdateProcessorTest, NumericFieldsTest, TestCoreDiscovery, MBeansHandlerTest, SolrPluginUtilsTest, DebugComponentTest, TestSchemaSimilarityResource, StandardRequestHandlerTest, DistributedTermsComponentTest, TestReRankQParserPlugin, TestCollapseQParserPlugin, SystemInfoHandlerTest, CurrencyFieldOpenExchangeTest, FieldFacetTest, TestFastLRUCache, TestCSVResponseWriter, SpatialFilterTest, CustomCollectionTest, TestFoldingMultitermQuery, BasicDistributedZk2Test, ReplicationFactorTest, SpellPossibilityIteratorTest, TestNonNRTOpen, SolrIndexSplitterTest, TestDocumentBuilder, AlternateDirectoryTest, EchoParamsTest, TestComponentsName, TestOrdValues, TestDistributedGrouping, ChaosMonkeySafeLeaderTest]
[junit4] Completed in 126.15s, 1 test, 1 failure <<< FAILURES!
[...truncated 1074 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:467: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:447: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:45: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:37: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:496: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1298: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:922: There were test failures: 415 suites, 1704 tests, 1 failure, 43 ignored (16 assumptions)
Total time: 106 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1688
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1688/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseParallelGC
All tests passed
Build Log:
[...truncated 44799 lines...]
-documentation-lint:
[echo] checking for broken html...
[jtidy] Checking for broken html (such as invalid tags)...
[delete] Deleting directory /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/jtidy_tmp
[echo] Checking for broken links...
[exec]
[exec] Crawl/parse...
[exec]
[exec] Verify...
[echo] Checking for missing docs...
[exec]
[exec] build/docs/facet/org/apache/lucene/facet/taxonomy/TaxonomyMergeUtils.html
[exec] missing Constructors: TaxonomyMergeUtils()
[exec]
[exec] Missing javadocs were found!
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:467: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:63: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build.xml:212: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build.xml:247: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:2338: exec returned: 1
Total time: 148 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any