You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/10/20 01:27:00 UTC
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_45) - Build # 7861
- Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/7861/
Java: 32bit/jdk1.7.0_45 -client -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 103 from http://127.0.0.1:35318/xy_/bz/collection1lastClient and got 102 from http://127.0.0.1:34262/xy_/bz/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 103 from http://127.0.0.1:35318/xy_/bz/collection1lastClient and got 102 from http://127.0.0.1:34262/xy_/bz/collection1
at __randomizedtesting.SeedInfo.seed([F04698EAC795B237:71A016F2B0CAD20B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1130)
at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:194)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Build Log:
[...truncated 10049 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
[junit4] 2> 97420 T443 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /xy_/bz
[junit4] 2> 97431 T443 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyNothingIsSafeTest-1382223984052
[junit4] 2> 97434 T443 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 97437 T444 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 97536 T443 oasc.ZkTestServer.run start zk server on port:58676
[junit4] 2> 97577 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 97608 T450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1577d0a name:ZooKeeperConnection Watcher:127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 97609 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 97610 T443 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 97633 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 97634 T452 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60b945 name:ZooKeeperConnection Watcher:127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 97634 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 97636 T443 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 97639 T443 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 97642 T443 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 97644 T443 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 97647 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 97647 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 97651 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 97652 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 97655 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 97655 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 97658 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 97658 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 97661 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 97661 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 97664 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 97664 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 97666 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 97667 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 97670 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 97670 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 97673 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 97674 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 97676 T443 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 97676 T443 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 97742 T443 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 97744 T443 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41444
[junit4] 2> 97745 T443 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 97745 T443 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 97746 T443 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301
[junit4] 2> 97746 T443 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/'
[junit4] 2> 97768 T443 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/solr.xml
[junit4] 2> 97816 T443 oasc.CoreContainer.<init> New CoreContainer 20085962
[junit4] 2> 97817 T443 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/]
[junit4] 2> 97818 T443 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 97818 T443 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 97818 T443 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 97819 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 97819 T443 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 97819 T443 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 97819 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 97820 T443 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 97820 T443 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 97821 T443 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 97821 T443 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 97822 T443 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58676/solr
[junit4] 2> 97829 T443 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 97830 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 97831 T463 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ec0c2 name:ZooKeeperConnection Watcher:127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 97832 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 97836 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 97837 T465 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e80e43 name:ZooKeeperConnection Watcher:127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 97838 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 97842 T443 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 97847 T443 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 97853 T443 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 97855 T443 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41444_xy_%2Fbz
[junit4] 2> 97858 T443 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41444_xy_%2Fbz
[junit4] 2> 97861 T443 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 97869 T443 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 97873 T443 oasc.Overseer.start Overseer (id=90585431024205827-127.0.0.1:41444_xy_%2Fbz-n_0000000000) starting
[junit4] 2> 97882 T443 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 97894 T467 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 97894 T443 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 97897 T443 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 97903 T443 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 97910 T466 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 97915 T468 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 97915 T468 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 97917 T468 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 99414 T466 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 99416 T466 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41444/xy_/bz",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41444_xy_%2Fbz",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 99416 T466 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 99422 T466 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 99480 T465 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 99917 T468 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 99917 T468 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1
[junit4] 2> 99918 T468 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 99918 T468 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 99919 T468 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 99920 T468 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1/'
[junit4] 2> 99921 T468 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1/lib/classes/' to classloader
[junit4] 2> 99921 T468 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1/lib/README' to classloader
[junit4] 2> 99964 T468 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
[junit4] 2> 100003 T468 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 100004 T468 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 100012 T468 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 100526 T468 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 100527 T468 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 100527 T468 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 100534 T468 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 100536 T468 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 100549 T468 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 100553 T468 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 100557 T468 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 100558 T468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 100558 T468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 100558 T468 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 100559 T468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 100559 T468 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 100560 T468 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 100566 T468 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/
[junit4] 2> 100566 T468 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@be73e9
[junit4] 2> 100567 T468 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data
[junit4] 2> 100567 T468 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index/
[junit4] 2> 100567 T468 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 100567 T468 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index
[junit4] 2> 100568 T468 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=42.689453125, floorSegmentMB=0.5654296875, forceMergeDeletesPctAllowed=3.6976139800581977, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 100623 T468 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 100623 T468 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 100626 T468 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 100626 T468 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 100626 T468 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 100626 T468 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 100627 T468 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 100627 T468 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 100627 T468 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 100627 T468 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 100628 T468 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 100628 T468 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 100628 T468 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 100629 T468 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 100629 T468 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 100629 T468 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 100630 T468 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 100632 T468 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 100634 T468 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 100635 T468 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 100635 T468 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 100636 T468 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 100636 T468 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 100636 T468 oass.SolrIndexSearcher.<init> Opening Searcher@62007f main
[junit4] 2> 100639 T469 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@62007f main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 100641 T468 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 100641 T468 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41444/xy_/bz collection:control_collection shard:shard1
[junit4] 2> 100646 T468 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 100655 T468 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 100658 T468 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 100658 T468 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 100660 T468 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41444/xy_/bz/collection1/
[junit4] 2> 100660 T468 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 100660 T468 oasc.SyncStrategy.syncToMe http://127.0.0.1:41444/xy_/bz/collection1/ has no replicas
[junit4] 2> 100660 T468 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41444/xy_/bz/collection1/ shard1
[junit4] 2> 100660 T468 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 100984 T466 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 100992 T465 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 101017 T468 oasc.ZkController.register We are http://127.0.0.1:41444/xy_/bz/collection1/ and leader is http://127.0.0.1:41444/xy_/bz/collection1/
[junit4] 2> 101017 T468 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41444/xy_/bz
[junit4] 2> 101017 T468 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 101017 T468 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 101017 T468 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 101020 T468 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 101021 T443 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 101021 T443 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 101049 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 101051 T472 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19b1550 name:ZooKeeperConnection Watcher:127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 101051 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 101052 T443 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 101056 T443 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 101125 T443 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 101126 T443 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41931
[junit4] 2> 101127 T443 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 101127 T443 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 101128 T443 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676
[junit4] 2> 101128 T443 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/'
[junit4] 2> 101139 T443 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/solr.xml
[junit4] 2> 101181 T443 oasc.CoreContainer.<init> New CoreContainer 6123382
[junit4] 2> 101182 T443 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/]
[junit4] 2> 101183 T443 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 101183 T443 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 101184 T443 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 101184 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 101184 T443 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 101184 T443 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 101185 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 101185 T443 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 101185 T443 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 101186 T443 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 101187 T443 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 101187 T443 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58676/solr
[junit4] 2> 101188 T443 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 101188 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 101190 T483 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@411e55 name:ZooKeeperConnection Watcher:127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 101190 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 101193 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 101194 T485 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d329f3 name:ZooKeeperConnection Watcher:127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 101194 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 101197 T443 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 102200 T443 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41931_xy_%2Fbz
[junit4] 2> 102271 T443 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41931_xy_%2Fbz
[junit4] 2> 102274 T472 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 102274 T465 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 102275 T485 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 102281 T486 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 102281 T486 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 102283 T486 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 102502 T466 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 102503 T466 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:41444/xy_/bz",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41444_xy_%2Fbz",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 102506 T466 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41931/xy_/bz",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41931_xy_%2Fbz",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 102506 T466 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 102507 T466 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 102510 T465 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 102510 T472 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 102510 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 103283 T486 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 103283 T486 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1
[junit4] 2> 103283 T486 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 103284 T486 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 103284 T486 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 103285 T486 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1/'
[junit4] 2> 103286 T486 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1/lib/classes/' to classloader
[junit4] 2> 103286 T486 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1/lib/README' to classloader
[junit4] 2> 103330 T486 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
[junit4] 2> 103368 T486 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 103369 T486 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 103377 T486 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 103771 T486 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 103772 T486 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 103772 T486 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 103778 T486 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 103781 T486 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 103797 T486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 103801 T486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 103805 T486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 103807 T486 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 103807 T486 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 103807 T486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 103809 T486 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 103809 T486 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 103809 T486 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 103809 T486 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/
[junit4] 2> 103810 T486 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@be73e9
[junit4] 2> 103810 T486 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1
[junit4] 2> 103811 T486 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index/
[junit4] 2> 103811 T486 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 103812 T486 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index
[junit4] 2> 103813 T486 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=42.689453125, floorSegmentMB=0.5654296875, forceMergeDeletesPctAllowed=3.6976139800581977, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 103817 T486 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 103817 T486 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 103822 T486 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 103822 T486 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 103823 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 103823 T486 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 103824 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 103824 T486 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 103825 T486 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 103825 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 103826 T486 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 103827 T486 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 103828 T486 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 103829 T486 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 103829 T486 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 103830 T486 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 103831 T486 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 103836 T486 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 103840 T486 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 103840 T486 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 103842 T486 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 103843 T486 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 103843 T486 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 103843 T486 oass.SolrIndexSearcher.<init> Opening Searcher@131b7cf main
[junit4] 2> 103847 T487 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@131b7cf main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 103849 T486 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 103849 T486 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41931/xy_/bz collection:collection1 shard:shard1
[junit4] 2> 103851 T486 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 103857 T486 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 103859 T486 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 103859 T486 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 103859 T486 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41931/xy_/bz/collection1/
[junit4] 2> 103859 T486 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 103860 T486 oasc.SyncStrategy.syncToMe http://127.0.0.1:41931/xy_/bz/collection1/ has no replicas
[junit4] 2> 103860 T486 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41931/xy_/bz/collection1/ shard1
[junit4] 2> 103860 T486 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 104016 T466 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 104024 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 104024 T465 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 104024 T472 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 104066 T486 oasc.ZkController.register We are http://127.0.0.1:41931/xy_/bz/collection1/ and leader is http://127.0.0.1:41931/xy_/bz/collection1/
[junit4] 2> 104067 T486 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41931/xy_/bz
[junit4] 2> 104067 T486 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 104067 T486 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 104068 T486 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 104070 T486 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 104071 T443 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 104071 T443 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 104146 T443 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 104148 T443 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40837
[junit4] 2> 104149 T443 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 104149 T443 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 104149 T443 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693
[junit4] 2> 104150 T443 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/'
[junit4] 2> 104164 T443 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/solr.xml
[junit4] 2> 104201 T443 oasc.CoreContainer.<init> New CoreContainer 2835059
[junit4] 2> 104202 T443 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/]
[junit4] 2> 104203 T443 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 104203 T443 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 104203 T443 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 104204 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 104204 T443 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 104204 T443 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 104204 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 104205 T443 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 104205 T443 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 104206 T443 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 104206 T443 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 104207 T443 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58676/solr
[junit4] 2> 104207 T443 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 104208 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 104210 T499 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fc606a name:ZooKeeperConnection Watcher:127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 104210 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 104214 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 104215 T501 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14876b4 name:ZooKeeperConnection Watcher:127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 104215 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 104220 T443 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 105224 T443 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40837_xy_%2Fbz
[junit4] 2> 105227 T443 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40837_xy_%2Fbz
[junit4] 2> 105232 T485 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 105232 T472 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 105232 T465 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 105232 T501 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 105244 T502 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 105245 T502 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 105247 T502 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 105530 T466 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 105531 T466 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:41931/xy_/bz",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41931_xy_%2Fbz",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 105535 T466 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40837/xy_/bz",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:40837_xy_%2Fbz",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 105535 T466 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 105536 T466 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 105539 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 105539 T472 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 105540 T465 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 105540 T501 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 106247 T502 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 106248 T502 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1
[junit4] 2> 106248 T502 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 106250 T502 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 106250 T502 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 106252 T502 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1/'
[junit4] 2> 106253 T502 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1/lib/classes/' to classloader
[junit4] 2> 106254 T502 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1/lib/README' to classloader
[junit4] 2> 106299 T502 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
[junit4] 2> 106337 T502 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 106338 T502 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 106346 T502 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 106740 T502 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 106740 T502 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 106741 T502 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 106746 T502 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 106749 T502 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 106762 T502 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 106766 T502 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 106771 T502 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 106772 T502 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 106773 T502 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 106773 T502 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 106774 T502 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 106774 T502 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 106775 T502 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 106775 T502 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/
[junit4] 2> 106775 T502 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@be73e9
[junit4] 2> 106776 T502 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2
[junit4] 2> 106777 T502 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index/
[junit4] 2> 106777 T502 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 106777 T502 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index
[junit4] 2> 106778 T502 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=42.689453125, floorSegmentMB=0.5654296875, forceMergeDeletesPctAllowed=3.6976139800581977, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 106781 T502 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 106781 T502 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 106783 T502 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 106783 T502 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 106784 T502 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 106784 T502 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 106784 T502 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 106784 T502 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 106785 T502 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 106785 T502 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 106785 T502 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 106786 T502 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 106786 T502 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 106787 T502 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 106787 T502 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 106787 T502 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 106788 T502 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 106790 T502 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 106792 T502 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 106792 T502 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 106793 T502 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 106794 T502 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 106794 T502 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 106794 T502 oass.SolrIndexSearcher.<init> Opening Searcher@191b873 main
[junit4] 2> 106798 T503 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@191b873 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 106801 T502 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 106801 T502 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40837/xy_/bz collection:collection1 shard:shard2
[junit4] 2> 106803 T502 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 106808 T502 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 106809 T502 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 106810 T502 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 106810 T502 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40837/xy_/bz/collection1/
[junit4] 2> 106810 T502 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 106810 T502 oasc.SyncStrategy.syncToMe http://127.0.0.1:40837/xy_/bz/collection1/ has no replicas
[junit4] 2> 106810 T502 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:40837/xy_/bz/collection1/ shard2
[junit4] 2> 106811 T502 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 107047 T466 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 107054 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 107054 T501 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 107054 T472 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 107055 T465 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 107066 T502 oasc.ZkController.register We are http://127.0.0.1:40837/xy_/bz/collection1/ and leader is http://127.0.0.1:40837/xy_/bz/collection1/
[junit4] 2> 107066 T502 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40837/xy_/bz
[junit4] 2> 107067 T502 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 107067 T502 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 107067 T502 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 107068 T502 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 107069 T443 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 107069 T443 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 107132 T443 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 107134 T443 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35331
[junit4] 2> 107135 T443 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 107135 T443 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 107135 T443 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690
[junit4] 2> 107135 T443 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/'
[junit4] 2> 107146 T443 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/solr.xml
[junit4] 2> 107184 T443 oasc.CoreContainer.<init> New CoreContainer 7674880
[junit4] 2> 107184 T443 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/]
[junit4] 2> 107185 T443 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 107186 T443 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 107186 T443 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 107186 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 107186 T443 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 107187 T443 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 107187 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 107187 T443 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 107187 T443 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 107189 T443 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 107189 T443 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 107189 T443 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58676/solr
[junit4] 2> 107190 T443 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 107190 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 107265 T515 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e5c27b name:ZooKeeperConnection Watcher:127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 107266 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 107268 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 107269 T517 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@894d7d name:ZooKeeperConnection Watcher:127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 107270 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 107274 T443 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 108277 T443 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35331_xy_%2Fbz
[junit4] 2> 108279 T443 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35331_xy_%2Fbz
[junit4] 2> 108281 T485 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 108282 T501 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 108282 T465 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 108281 T472 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 108282 T517 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 108293 T518 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 108293 T518 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 108295 T518 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 108559 T466 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 108560 T466 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:40837/xy_/bz",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:40837_xy_%2Fbz",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 108564 T466 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35331/xy_/bz",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35331_xy_%2Fbz",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node3"}
[junit4] 2> 108564 T466 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 108564 T466 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 108567 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 108567 T465 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 108567 T517 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 108567 T472 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 108567 T501 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 109296 T518 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 109296 T518 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1
[junit4] 2> 109296 T518 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 109297 T518 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 109297 T518 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 109298 T518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1/'
[junit4] 2> 109299 T518 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1/lib/classes/' to classloader
[junit4] 2> 109300 T518 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1/lib/README' to classloader
[junit4] 2> 109341 T518 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
[junit4] 2> 109388 T518 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 109389 T518 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 109396 T518 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 109783 T518 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 109784 T518 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 109785 T518 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 109791 T518 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 109793 T518 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 109804 T518 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 109807 T518 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 109810 T518 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 109811 T518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 109811 T518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 109811 T518 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 109812 T518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 109812 T518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 109813 T518 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 109813 T518 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/
[junit4] 2> 109813 T518 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@be73e9
[junit4] 2> 109814 T518 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3
[junit4] 2> 109814 T518 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index/
[junit4] 2> 109814 T518 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 109814 T518 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index
[junit4] 2> 109815 T518 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=42.689453125, floorSegmentMB=0.5654296875, forceMergeDeletesPctAllowed=3.6976139800581977, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 109817 T518 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index,segFN=segments_1,generation=1}
[junit4] 2> 109817 T518 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 109819 T518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 109820 T518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 109820 T518 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 109820 T518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 109820 T518 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 109821 T518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 109821 T518 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 109821 T518 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 109821 T518 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 109822 T518 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 109823 T518 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 109823 T518 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 109823 T518 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 109824 T518 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 109824 T518 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 109826 T518 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 109828 T518 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 109829 T518 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 109829 T518 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 109830 T518 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index,segFN=segments_1,generation=1}
[junit4] 2> 109830 T518 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 109831 T518 oass.SolrIndexSearcher.<init> Opening Searcher@14234b0 main
[junit4] 2> 109835 T519 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14234b0 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 109837 T518 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 109837 T518 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35331/xy_/bz collection:collection1 shard:shard1
[junit4] 2> 109840 T518 oasc.ZkController.register We are http://127.0.0.1:35331/xy_/bz/collection1/ and leader is http://127.0.0.1:41931/xy_/bz/collection1/
[junit4] 2> 109841 T518 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35331/xy_/bz
[junit4] 2> 109841 T518 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 109841 T518 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C291 name=collection1 org.apache.solr.core.SolrCore@9a575b url=http://127.0.0.1:35331/xy_/bz/collection1 node=127.0.0.1:35331_xy_%2Fbz C291_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:35331/xy_/bz, core=collection1, node_name=127.0.0.1:35331_xy_%2Fbz}
[junit4] 2> 109844 T520 C291 P35331 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 109844 T518 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 109845 T520 C291 P35331 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 109845 T520 C291 P35331 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 109845 T520 C291 P35331 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 109846 T443 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 109846 T443 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 109919 T443 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 109921 T443 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52309
[junit4] 2> 109921 T443 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 109922 T443 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 109922 T443 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1382223996467
[junit4] 2> 109923 T443 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1382223996467/'
[junit4] 2> 109941 T443 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1382223996467/solr.xml
[junit4] 2> 109990 T443 oasc.CoreContainer.<init> New CoreContainer 23229910
[junit4] 2> 109991 T443 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1382223996467/]
[junit4] 2> 109992 T443 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 109992 T443 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 109992 T443 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 109993 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 109993 T443 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 109993 T443 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 109994 T443 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 109994 T443 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 109994 T443 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 109995 T443 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 109996 T443 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 109996 T443 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58676/solr
[junit4] 2> 109996 T443 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 109997 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 109998 T532 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12f23a2 name:ZooKeeperConnection Watcher:127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 109999 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 110001 T443 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 110002 T534 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12ac54 name:ZooKeeperConnection Watcher:127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 110002 T443 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4]
[...truncated too long message...]
44_xy_%2Fbz"}}}},
[junit4] 1> "maxShardsPerNode":"1",
[junit4] 1> "router":{"name":"compositeId"},
[junit4] 1> "replicationFactor":"1"},
[junit4] 1> "control_collection":{
[junit4] 1> "shards":{"shard1":{
[junit4] 1> "range":"80000000-7fffffff",
[junit4] 1> "state":"active",
[junit4] 1> "replicas":{"core_node1":{
[junit4] 1> "state":"active",
[junit4] 1> "base_url":"http://127.0.0.1:41444/xy_/bz",
[junit4] 1> "core":"collection1",
[junit4] 1> "node_name":"127.0.0.1:41444_xy_%2Fbz",
[junit4] 1> "leader":"true"}}}},
[junit4] 1> "maxShardsPerNode":"1",
[junit4] 1> "router":{"name":"compositeId"},
[junit4] 1> "replicationFactor":"1"}}
[junit4] 1> /solr/aliases.json (0)
[junit4] 1> /solr/live_nodes (8)
[junit4] 1> /solr/live_nodes/127.0.0.1:40837_xy_%2Fbz (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:41931_xy_%2Fbz (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:41444_xy_%2Fbz (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:36944_xy_%2Fbz (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:35318_xy_%2Fbz (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:35331_xy_%2Fbz (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:52309_xy_%2Fbz (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:34262_xy_%2Fbz (0)
[junit4] 1> /solr/overseer (3)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1> /solr/overseer/queue (0)
[junit4] 1> /solr/overseer/queue-work (0)
[junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4] 1> /solr/collections (2)
[junit4] 1> /solr/collections/collection1 (3)
[junit4] 1> DATA:
[junit4] 1> {"configName":"conf1"}
[junit4] 1> /solr/collections/collection1/shards (0)
[junit4] 1> /solr/collections/collection1/leader_elect (2)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (4)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/90585431024205838-core_node5-n_0000000002 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/90585431024205842-core_node7-n_0000000003 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/90585431024205834-core_node3-n_0000000001 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/90585431024205830-core_node1-n_0000000000 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2 (1)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election (3)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/90585431024205840-core_node6-n_0000000002 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/90585431024205836-core_node4-n_0000000001 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/90585431024205832-core_node2-n_0000000000 (0)
[junit4] 1> /solr/collections/collection1/leaders (2)
[junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "core":"collection1",
[junit4] 1> "node_name":"127.0.0.1:41931_xy_%2Fbz",
[junit4] 1> "base_url":"http://127.0.0.1:41931/xy_/bz"}
[junit4] 1> /solr/collections/collection1/leaders/shard2 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "core":"collection1",
[junit4] 1> "node_name":"127.0.0.1:40837_xy_%2Fbz",
[junit4] 1> "base_url":"http://127.0.0.1:40837/xy_/bz"}
[junit4] 1> /solr/collections/control_collection (3)
[junit4] 1> DATA:
[junit4] 1> {"configName":"conf1"}
[junit4] 1> /solr/collections/control_collection/shards (0)
[junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/90585431024205827-core_node1-n_0000000000 (0)
[junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "core":"collection1",
[junit4] 1> "node_name":"127.0.0.1:41444_xy_%2Fbz",
[junit4] 1> "base_url":"http://127.0.0.1:41444/xy_/bz"}
[junit4] 1> /solr/overseer_elect (2)
[junit4] 1> /solr/overseer_elect/election (8)
[junit4] 1> /solr/overseer_elect/election/90585431024205838-127.0.0.1:35318_xy_%2Fbz-n_0000000005 (0)
[junit4] 1> /solr/overseer_elect/election/90585431024205842-127.0.0.1:34262_xy_%2Fbz-n_0000000007 (0)
[junit4] 1> /solr/overseer_elect/election/90585431024205827-127.0.0.1:41444_xy_%2Fbz-n_0000000000 (0)
[junit4] 1> /solr/overseer_elect/election/90585431024205840-127.0.0.1:36944_xy_%2Fbz-n_0000000006 (0)
[junit4] 1> /solr/overseer_elect/election/90585431024205830-127.0.0.1:41931_xy_%2Fbz-n_0000000001 (0)
[junit4] 1> /solr/overseer_elect/election/90585431024205834-127.0.0.1:35331_xy_%2Fbz-n_0000000003 (0)
[junit4] 1> /solr/overseer_elect/election/90585431024205836-127.0.0.1:52309_xy_%2Fbz-n_0000000004 (0)
[junit4] 1> /solr/overseer_elect/election/90585431024205832-127.0.0.1:40837_xy_%2Fbz-n_0000000002 (0)
[junit4] 1> /solr/overseer_elect/leader (0)
[junit4] 1> DATA:
[junit4] 1> {"id":"90585431024205827-127.0.0.1:41444_xy_%2Fbz-n_0000000000"}
[junit4] 1> /zookeeper (1)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1>
[junit4] 2> 156498 T569 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.method=testDistribSearch -Dtests.seed=F04698EAC795B237 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_LY -Dtests.timezone=America/Argentina/San_Juan -Dtests.file.encoding=UTF-8
[junit4] FAILURE 59.1s J1 | ChaosMonkeyNothingIsSafeTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 103 from http://127.0.0.1:35318/xy_/bz/collection1lastClient and got 102 from http://127.0.0.1:34262/xy_/bz/collection1
[junit4] > at __randomizedtesting.SeedInfo.seed([F04698EAC795B237:71A016F2B0CAD20B]:0)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1130)
[junit4] > at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:194)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 156516 T443 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 59102 T442 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> 157622 T588 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: test params are: codec=Appending, sim=DefaultSimilarity, locale=ar_LY, timezone=America/Argentina/San_Juan
[junit4] 2> NOTE: Linux 3.8.0-31-generic i386/Oracle Corporation 1.7.0_45 (32-bit)/cpus=8,threads=1,free=105194416,total=205258752
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrXmlPersistence, TestFieldTypeResource, SuggesterWFSTTest, TestDFRSimilarityFactory, XsltUpdateRequestHandlerTest, RequestHandlersTest, TestManagedSchemaFieldResource, TestIndexSearcher, TestSolrQueryParserResource, TestStressLucene, SimplePostToolTest, TestBinaryResponseWriter, TestInfoStreamLogging, TestNumberUtils, TestCollationField, TestIBSimilarityFactory, OpenExchangeRatesOrgProviderTest, TestAtomicUpdateErrorCases, DirectSolrConnectionTest, PreAnalyzedUpdateProcessorTest, SpellCheckComponentTest, TestStressReorder, TestStressRecovery, NumericFieldsTest, ChaosMonkeyNothingIsSafeTest]
[junit4] Completed on J1 in 60.39s, 1 test, 1 failure <<< FAILURES!
[...truncated 825 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:428: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:408: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:483: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1263: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:906: There were test failures: 327 suites, 1453 tests, 1 failure, 35 ignored (6 assumptions)
Total time: 46 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_45 -client -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_45) - Build #
7861 - Failure!
Posted by Mark Miller <ma...@gmail.com>.
Pretty rare, but I've seen one locally too - my initial guess is that
streaming updates has changed some timing or something that is exposing an
issue.
On Sat, Oct 19, 2013 at 7:27 PM, Policeman Jenkins Server <
jenkins@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/7861/
> Java: 32bit/jdk1.7.0_45 -client -XX:+UseParallelGC
>
> 1 tests failed.
> REGRESSION:
> org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.testDistribSearch
>
> Error Message:
> shard1 is not consistent. Got 103 from
> http://127.0.0.1:35318/xy_/bz/collection1lastClient and got 102 from
> http://127.0.0.1:34262/xy_/bz/collection1
>
> Stack Trace:
> java.lang.AssertionError: shard1 is not consistent. Got 103 from
> http://127.0.0.1:35318/xy_/bz/collection1lastClient and got 102 from
> http://127.0.0.1:34262/xy_/bz/collection1
> at
> __randomizedtesting.SeedInfo.seed([F04698EAC795B237:71A016F2B0CAD20B]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1130)
> at
> org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:194)
> at
> org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at java.lang.Thread.run(Thread.java:744)
>
>
>
>
> Build Log:
> [...truncated 10049 lines...]
> [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
> [junit4] 2> 97420 T443
> oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext
> system property: /xy_/bz
> [junit4] 2> 97431 T443 oas.SolrTestCaseJ4.setUp ###Starting
> testDistribSearch
> [junit4] 2> Creating dataDir:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyNothingIsSafeTest-1382223984052
> [junit4] 2> 97434 T443 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4] 2> 97437 T444 oasc.ZkTestServer$ZKServerMain.runFromConfig
> Starting server
> [junit4] 2> 97536 T443 oasc.ZkTestServer.run start zk server on
> port:58676
> [junit4] 2> 97577 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 97608 T450 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@1577d0aname:ZooKeeperConnection Watcher:
> 127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 97609 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 97610 T443 oascc.SolrZkClient.makePath makePath: /solr
> [junit4] 2> 97633 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 97634 T452 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@60b945name:ZooKeeperConnection Watcher:
> 127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 97634 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 97636 T443 oascc.SolrZkClient.makePath makePath:
> /collections/collection1
> [junit4] 2> 97639 T443 oascc.SolrZkClient.makePath makePath:
> /collections/collection1/shards
> [junit4] 2> 97642 T443 oascc.SolrZkClient.makePath makePath:
> /collections/control_collection
> [junit4] 2> 97644 T443 oascc.SolrZkClient.makePath makePath:
> /collections/control_collection/shards
> [junit4] 2> 97647 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml
> to /configs/conf1/solrconfig.xml
> [junit4] 2> 97647 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/solrconfig.xml
> [junit4] 2> 97651 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml
> to /configs/conf1/schema.xml
> [junit4] 2> 97652 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/schema.xml
> [junit4] 2> 97655 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
> to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 97655 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 97658 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt
> to /configs/conf1/stopwords.txt
> [junit4] 2> 97658 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/stopwords.txt
> [junit4] 2> 97661 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt
> to /configs/conf1/protwords.txt
> [junit4] 2> 97661 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/protwords.txt
> [junit4] 2> 97664 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml
> to /configs/conf1/currency.xml
> [junit4] 2> 97664 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/currency.xml
> [junit4] 2> 97666 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json
> to /configs/conf1/open-exchange-rates.json
> [junit4] 2> 97667 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/open-exchange-rates.json
> [junit4] 2> 97670 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
> to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 97670 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 97673 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt
> to /configs/conf1/old_synonyms.txt
> [junit4] 2> 97674 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/old_synonyms.txt
> [junit4] 2> 97676 T443 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt
> to /configs/conf1/synonyms.txt
> [junit4] 2> 97676 T443 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/synonyms.txt
> [junit4] 2> 97742 T443 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 97744 T443 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:41444
> [junit4] 2> 97745 T443 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
> [junit4] 2> 97745 T443 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
> [junit4] 2> 97746 T443 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301
> [junit4] 2> 97746 T443 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/'
> [junit4] 2> 97768 T443 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/solr.xml
> [junit4] 2> 97816 T443 oasc.CoreContainer.<init> New CoreContainer
> 20085962
> [junit4] 2> 97817 T443 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/]
> [junit4] 2> 97818 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 120000
> [junit4] 2> 97818 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to: http://
> [junit4] 2> 97818 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
> [junit4] 2> 97819 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
> [junit4] 2> 97819 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
> [junit4] 2> 97819 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
> [junit4] 2> 97819 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
> [junit4] 2> 97820 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
> [junit4] 2> 97820 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
> [junit4] 2> 97821 T443 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 97821 T443 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 97822 T443 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:58676/solr
> [junit4] 2> 97829 T443 oasc.ZkController.checkChrootPath zkHost
> includes chroot
> [junit4] 2> 97830 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 97831 T463 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@ec0c2name:ZooKeeperConnection Watcher:
> 127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 97832 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 97836 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 97837 T465 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@e80e43name:ZooKeeperConnection Watcher:
> 127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 97838 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 97842 T443 oascc.SolrZkClient.makePath makePath:
> /overseer/queue
> [junit4] 2> 97847 T443 oascc.SolrZkClient.makePath makePath:
> /overseer/collection-queue-work
> [junit4] 2> 97853 T443 oascc.SolrZkClient.makePath makePath:
> /live_nodes
> [junit4] 2> 97855 T443 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41444_xy_%2Fbz
> [junit4] 2> 97858 T443 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:41444_xy_%2Fbz
> [junit4] 2> 97861 T443 oascc.SolrZkClient.makePath makePath:
> /overseer_elect/election
> [junit4] 2> 97869 T443 oascc.SolrZkClient.makePath makePath:
> /overseer_elect/leader
> [junit4] 2> 97873 T443 oasc.Overseer.start Overseer
> (id=90585431024205827-127.0.0.1:41444_xy_%2Fbz-n_0000000000) starting
> [junit4] 2> 97882 T443 oascc.SolrZkClient.makePath makePath:
> /overseer/queue-work
> [junit4] 2> 97894 T467 oasc.OverseerCollectionProcessor.run Process
> current queue of collection creations
> [junit4] 2> 97894 T443 oascc.SolrZkClient.makePath makePath:
> /clusterstate.json
> [junit4] 2> 97897 T443 oascc.SolrZkClient.makePath makePath:
> /aliases.json
> [junit4] 2> 97903 T443
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
> [junit4] 2> 97910 T466 oasc.Overseer$ClusterStateUpdater.run Starting
> to work on the main queue
> [junit4] 2> 97915 T468 oasc.ZkController.publish publishing
> core=collection1 state=down
> [junit4] 2> 97915 T468 oasc.ZkController.publish numShards not found
> on descriptor - reading it from system property
> [junit4] 2> 97917 T468 oasc.ZkController.waitForCoreNodeName look for
> our core node name
> [junit4] 2> 99414 T466 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 99416 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Update state numShards=1 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:41444/xy_/bz",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:41444_xy_%2Fbz",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"control_collection",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 99416 T466
> oasc.Overseer$ClusterStateUpdater.createCollection Create collection
> control_collection with shards [shard1]
> [junit4] 2> 99422 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Assigning new node to shard shard=shard1
> [junit4] 2> 99480 T465 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 99917 T468 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for collection1
> [junit4] 2> 99917 T468 oasc.CoreContainer.create Creating SolrCore
> 'collection1' using instanceDir:
> ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1
> [junit4] 2> 99918 T468 oasc.ZkController.createCollectionZkNode Check
> for collection zkNode:control_collection
> [junit4] 2> 99918 T468 oasc.ZkController.createCollectionZkNode
> Collection zkNode exists
> [junit4] 2> 99919 T468 oasc.ZkController.readConfigName Load
> collection config from:/collections/control_collection
> [junit4] 2> 99920 T468 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1/'
> [junit4] 2> 99921 T468 oasc.SolrResourceLoader.replaceClassLoader
> Adding
> 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1/lib/classes/'
> to classloader
> [junit4] 2> 99921 T468 oasc.SolrResourceLoader.replaceClassLoader
> Adding
> 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1/lib/README'
> to classloader
> [junit4] 2> 99964 T468 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: LUCENE_46
> [junit4] 2> 100003 T468 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
> [junit4] 2> 100004 T468 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
> [junit4] 2> 100012 T468 oass.IndexSchema.readSchema [collection1]
> Schema name=test
> [junit4] 2> 100526 T468 oass.ByteField.init WARN ByteField is
> deprecated and will be removed in 5.0. You should use TrieIntField instead.
> [junit4] 2> 100527 T468 oass.ShortField.init WARN ShortField is
> deprecated and will be removed in 5.0. You should use TrieIntField instead.
> [junit4] 2> 100527 T468 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4] 2> 100534 T468 oass.IndexSchema.readSchema default search
> field in schema is text
> [junit4] 2> 100536 T468 oass.IndexSchema.readSchema unique key field:
> id
> [junit4] 2> 100549 T468 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
> [junit4] 2> 100553 T468 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
> [junit4] 2> 100557 T468 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 100558 T468
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
> [junit4] 2> 100558 T468
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
> [junit4] 2> 100558 T468 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 100559 T468
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
> [junit4] 2> 100559 T468
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
> [junit4] 2> 100560 T468 oasc.SolrCore.initDirectoryFactory
> solr.StandardDirectoryFactory
> [junit4] 2> 100566 T468 oasc.SolrCore.<init> [collection1] Opening
> new SolrCore at
> ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1382223984301/collection1/,
> dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/
> [junit4] 2> 100566 T468 oasc.JmxMonitoredMap.<init> JMX monitoring is
> enabled. Adding Solr mbeans to JMX Server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@be73e9
> [junit4] 2> 100567 T468 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data
> [junit4] 2> 100567 T468 oasc.SolrCore.getNewIndexDir New index
> directory detected: old=null
> new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index/
> [junit4] 2> 100567 T468 oasc.SolrCore.initIndex WARN [collection1]
> Solr index directory
> './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index'
> doesn't exist. Creating new index...
> [junit4] 2> 100567 T468 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index
> [junit4] 2> 100568 T468 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=29,
> maxMergedSegmentMB=42.689453125, floorSegmentMB=0.5654296875,
> forceMergeDeletesPctAllowed=3.6976139800581977, segmentsPerTier=26.0,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
> [junit4] 2> 100623 T468 oasc.SolrDeletionPolicy.onCommit
> SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2>
> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index,segFN=segments_1,generation=1}
> [junit4] 2> 100623 T468 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
> [junit4] 2> 100626 T468 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 100626 T468 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 100626 T468 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "dedupe"
> [junit4] 2> 100626 T468 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 100627 T468 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "stored_sig"
> [junit4] 2> 100627 T468 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 100627 T468 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 100627 T468 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 100628 T468 oasc.SolrCore.loadUpdateProcessorChains no
> updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 100628 T468 oasc.RequestHandlers.initHandlersFromConfig
> created standard: solr.StandardRequestHandler
> [junit4] 2> 100628 T468 oasc.RequestHandlers.initHandlersFromConfig
> created /get: solr.RealTimeGetHandler
> [junit4] 2> 100629 T468 oasc.RequestHandlers.initHandlersFromConfig
> adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 100629 T468 oasc.RequestHandlers.initHandlersFromConfig
> created /replication: solr.ReplicationHandler
> [junit4] 2> 100629 T468 oasc.RequestHandlers.initHandlersFromConfig
> created /update: solr.UpdateRequestHandler
> [junit4] 2> 100630 T468 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 100632 T468 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
> [junit4] 2> 100634 T468 oasu.CommitTracker.<init> Hard AutoCommit:
> disabled
> [junit4] 2> 100635 T468 oasu.CommitTracker.<init> Soft AutoCommit:
> disabled
> [junit4] 2> 100635 T468 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class
> org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy:
> minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648,
> maxMergeSizeForForcedMerge=9223372036854775807,
> calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
> [junit4] 2> 100636 T468 oasc.SolrDeletionPolicy.onInit
> SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2>
> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/control/data/index,segFN=segments_1,generation=1}
> [junit4] 2> 100636 T468 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
> [junit4] 2> 100636 T468 oass.SolrIndexSearcher.<init> Opening
> Searcher@62007f main
> [junit4] 2> 100639 T469 oasc.SolrCore.registerSearcher [collection1]
> Registered new searcher Searcher@62007fmain{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 100641 T468 oasc.CoreContainer.registerCore registering
> core: collection1
> [junit4] 2> 100641 T468 oasc.ZkController.register Register replica -
> core:collection1 address:http://127.0.0.1:41444/xy_/bzcollection:control_collection shard:shard1
> [junit4] 2> 100646 T468 oascc.SolrZkClient.makePath makePath:
> /collections/control_collection/leader_elect/shard1/election
> [junit4] 2> 100655 T468
> oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process
> for shard shard1
> [junit4] 2> 100658 T468
> oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas
> found to continue.
> [junit4] 2> 100658 T468
> oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader -
> try and sync
> [junit4] 2> 100660 T468 oasc.SyncStrategy.sync Sync replicas to
> http://127.0.0.1:41444/xy_/bz/collection1/
> [junit4] 2> 100660 T468 oasc.SyncStrategy.syncReplicas Sync Success -
> now sync replicas to me
> [junit4] 2> 100660 T468 oasc.SyncStrategy.syncToMe
> http://127.0.0.1:41444/xy_/bz/collection1/ has no replicas
> [junit4] 2> 100660 T468
> oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader:
> http://127.0.0.1:41444/xy_/bz/collection1/ shard1
> [junit4] 2> 100660 T468 oascc.SolrZkClient.makePath makePath:
> /collections/control_collection/leaders/shard1
> [junit4] 2> 100984 T466 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 100992 T465 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 101017 T468 oasc.ZkController.register We are
> http://127.0.0.1:41444/xy_/bz/collection1/ and leader is
> http://127.0.0.1:41444/xy_/bz/collection1/
> [junit4] 2> 101017 T468 oasc.ZkController.register No LogReplay
> needed for core=collection1 baseURL=http://127.0.0.1:41444/xy_/bz
> [junit4] 2> 101017 T468 oasc.ZkController.checkRecovery I am the
> leader, no recovery necessary
> [junit4] 2> 101017 T468 oasc.ZkController.publish publishing
> core=collection1 state=active
> [junit4] 2> 101017 T468 oasc.ZkController.publish numShards not found
> on descriptor - reading it from system property
> [junit4] 2> 101020 T468 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 101021 T443 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
> [junit4] 2> 101021 T443 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
> [junit4] 2> 101049 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 101051 T472 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@19b1550name:ZooKeeperConnection Watcher:
> 127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 101051 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 101052 T443
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
> [junit4] 2> 101056 T443 oasc.ChaosMonkey.monkeyLog monkey: init -
> expire sessions:true cause connection loss:true
> [junit4] 2> 101125 T443 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 101126 T443 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:41931
> [junit4] 2> 101127 T443 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
> [junit4] 2> 101127 T443 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
> [junit4] 2> 101128 T443 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676
> [junit4] 2> 101128 T443 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/'
> [junit4] 2> 101139 T443 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/solr.xml
> [junit4] 2> 101181 T443 oasc.CoreContainer.<init> New CoreContainer
> 6123382
> [junit4] 2> 101182 T443 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/]
> [junit4] 2> 101183 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 120000
> [junit4] 2> 101183 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to: http://
> [junit4] 2> 101184 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
> [junit4] 2> 101184 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
> [junit4] 2> 101184 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
> [junit4] 2> 101184 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
> [junit4] 2> 101185 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
> [junit4] 2> 101185 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
> [junit4] 2> 101185 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
> [junit4] 2> 101186 T443 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 101187 T443 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 101187 T443 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:58676/solr
> [junit4] 2> 101188 T443 oasc.ZkController.checkChrootPath zkHost
> includes chroot
> [junit4] 2> 101188 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 101190 T483 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@411e55name:ZooKeeperConnection Watcher:
> 127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 101190 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 101193 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 101194 T485 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@1d329f3name:ZooKeeperConnection Watcher:
> 127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 101194 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 101197 T443
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
> [junit4] 2> 102200 T443 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41931_xy_%2Fbz
> [junit4] 2> 102271 T443 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:41931_xy_%2Fbz
> [junit4] 2> 102274 T472 oascc.ZkStateReader$3.process Updating live
> nodes... (2)
> [junit4] 2> 102274 T465 oascc.ZkStateReader$3.process Updating live
> nodes... (2)
> [junit4] 2> 102275 T485 oascc.ZkStateReader$3.process Updating live
> nodes... (2)
> [junit4] 2> 102281 T486 oasc.ZkController.publish publishing
> core=collection1 state=down
> [junit4] 2> 102281 T486 oasc.ZkController.publish numShards not found
> on descriptor - reading it from system property
> [junit4] 2> 102283 T486 oasc.ZkController.waitForCoreNodeName look
> for our core node name
> [junit4] 2> 102502 T466 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 102503 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Update state numShards=1 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"http://127.0.0.1:41444/xy_/bz",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:41444_xy_%2Fbz",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"control_collection",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 102506 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:41931/xy_/bz",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:41931_xy_%2Fbz",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 102506 T466
> oasc.Overseer$ClusterStateUpdater.createCollection Create collection
> collection1 with shards [shard1, shard2]
> [junit4] 2> 102507 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Assigning new node to shard shard=shard1
> [junit4] 2> 102510 T465 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 102510 T472 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 102510 T485 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 103283 T486 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for collection1
> [junit4] 2> 103283 T486 oasc.CoreContainer.create Creating SolrCore
> 'collection1' using instanceDir:
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1
> [junit4] 2> 103283 T486 oasc.ZkController.createCollectionZkNode
> Check for collection zkNode:collection1
> [junit4] 2> 103284 T486 oasc.ZkController.createCollectionZkNode
> Collection zkNode exists
> [junit4] 2> 103284 T486 oasc.ZkController.readConfigName Load
> collection config from:/collections/collection1
> [junit4] 2> 103285 T486 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1/'
> [junit4] 2> 103286 T486 oasc.SolrResourceLoader.replaceClassLoader
> Adding
> 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1/lib/classes/'
> to classloader
> [junit4] 2> 103286 T486 oasc.SolrResourceLoader.replaceClassLoader
> Adding
> 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1/lib/README'
> to classloader
> [junit4] 2> 103330 T486 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: LUCENE_46
> [junit4] 2> 103368 T486 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
> [junit4] 2> 103369 T486 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
> [junit4] 2> 103377 T486 oass.IndexSchema.readSchema [collection1]
> Schema name=test
> [junit4] 2> 103771 T486 oass.ByteField.init WARN ByteField is
> deprecated and will be removed in 5.0. You should use TrieIntField instead.
> [junit4] 2> 103772 T486 oass.ShortField.init WARN ShortField is
> deprecated and will be removed in 5.0. You should use TrieIntField instead.
> [junit4] 2> 103772 T486 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4] 2> 103778 T486 oass.IndexSchema.readSchema default search
> field in schema is text
> [junit4] 2> 103781 T486 oass.IndexSchema.readSchema unique key field:
> id
> [junit4] 2> 103797 T486 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
> [junit4] 2> 103801 T486 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
> [junit4] 2> 103805 T486 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 103807 T486
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
> [junit4] 2> 103807 T486
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
> [junit4] 2> 103807 T486 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 103809 T486
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
> [junit4] 2> 103809 T486
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
> [junit4] 2> 103809 T486 oasc.SolrCore.initDirectoryFactory
> solr.StandardDirectoryFactory
> [junit4] 2> 103809 T486 oasc.SolrCore.<init> [collection1] Opening
> new SolrCore at
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1382223987676/collection1/,
> dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/
> [junit4] 2> 103810 T486 oasc.JmxMonitoredMap.<init> JMX monitoring is
> enabled. Adding Solr mbeans to JMX Server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@be73e9
> [junit4] 2> 103810 T486 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1
> [junit4] 2> 103811 T486 oasc.SolrCore.getNewIndexDir New index
> directory detected: old=null
> new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index/
> [junit4] 2> 103811 T486 oasc.SolrCore.initIndex WARN [collection1]
> Solr index directory
> './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index'
> doesn't exist. Creating new index...
> [junit4] 2> 103812 T486 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index
> [junit4] 2> 103813 T486 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=29,
> maxMergedSegmentMB=42.689453125, floorSegmentMB=0.5654296875,
> forceMergeDeletesPctAllowed=3.6976139800581977, segmentsPerTier=26.0,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
> [junit4] 2> 103817 T486 oasc.SolrDeletionPolicy.onCommit
> SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2>
> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index,segFN=segments_1,generation=1}
> [junit4] 2> 103817 T486 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
> [junit4] 2> 103822 T486 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 103822 T486 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 103823 T486 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "dedupe"
> [junit4] 2> 103823 T486 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 103824 T486 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "stored_sig"
> [junit4] 2> 103824 T486 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 103825 T486 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 103825 T486 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 103826 T486 oasc.SolrCore.loadUpdateProcessorChains no
> updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 103827 T486 oasc.RequestHandlers.initHandlersFromConfig
> created standard: solr.StandardRequestHandler
> [junit4] 2> 103828 T486 oasc.RequestHandlers.initHandlersFromConfig
> created /get: solr.RealTimeGetHandler
> [junit4] 2> 103829 T486 oasc.RequestHandlers.initHandlersFromConfig
> adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 103829 T486 oasc.RequestHandlers.initHandlersFromConfig
> created /replication: solr.ReplicationHandler
> [junit4] 2> 103830 T486 oasc.RequestHandlers.initHandlersFromConfig
> created /update: solr.UpdateRequestHandler
> [junit4] 2> 103831 T486 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 103836 T486 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
> [junit4] 2> 103840 T486 oasu.CommitTracker.<init> Hard AutoCommit:
> disabled
> [junit4] 2> 103840 T486 oasu.CommitTracker.<init> Soft AutoCommit:
> disabled
> [junit4] 2> 103842 T486 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class
> org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy:
> minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648,
> maxMergeSizeForForcedMerge=9223372036854775807,
> calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
> [junit4] 2> 103843 T486 oasc.SolrDeletionPolicy.onInit
> SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2>
> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty1/index,segFN=segments_1,generation=1}
> [junit4] 2> 103843 T486 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
> [junit4] 2> 103843 T486 oass.SolrIndexSearcher.<init> Opening
> Searcher@131b7cf main
> [junit4] 2> 103847 T487 oasc.SolrCore.registerSearcher [collection1]
> Registered new searcher Searcher@131b7cfmain{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 103849 T486 oasc.CoreContainer.registerCore registering
> core: collection1
> [junit4] 2> 103849 T486 oasc.ZkController.register Register replica -
> core:collection1 address:http://127.0.0.1:41931/xy_/bzcollection:collection1 shard:shard1
> [junit4] 2> 103851 T486 oascc.SolrZkClient.makePath makePath:
> /collections/collection1/leader_elect/shard1/election
> [junit4] 2> 103857 T486
> oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process
> for shard shard1
> [junit4] 2> 103859 T486
> oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas
> found to continue.
> [junit4] 2> 103859 T486
> oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader -
> try and sync
> [junit4] 2> 103859 T486 oasc.SyncStrategy.sync Sync replicas to
> http://127.0.0.1:41931/xy_/bz/collection1/
> [junit4] 2> 103859 T486 oasc.SyncStrategy.syncReplicas Sync Success -
> now sync replicas to me
> [junit4] 2> 103860 T486 oasc.SyncStrategy.syncToMe
> http://127.0.0.1:41931/xy_/bz/collection1/ has no replicas
> [junit4] 2> 103860 T486
> oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader:
> http://127.0.0.1:41931/xy_/bz/collection1/ shard1
> [junit4] 2> 103860 T486 oascc.SolrZkClient.makePath makePath:
> /collections/collection1/leaders/shard1
> [junit4] 2> 104016 T466 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 104024 T485 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 104024 T465 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 104024 T472 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 104066 T486 oasc.ZkController.register We are
> http://127.0.0.1:41931/xy_/bz/collection1/ and leader is
> http://127.0.0.1:41931/xy_/bz/collection1/
> [junit4] 2> 104067 T486 oasc.ZkController.register No LogReplay
> needed for core=collection1 baseURL=http://127.0.0.1:41931/xy_/bz
> [junit4] 2> 104067 T486 oasc.ZkController.checkRecovery I am the
> leader, no recovery necessary
> [junit4] 2> 104067 T486 oasc.ZkController.publish publishing
> core=collection1 state=active
> [junit4] 2> 104068 T486 oasc.ZkController.publish numShards not found
> on descriptor - reading it from system property
> [junit4] 2> 104070 T486 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 104071 T443 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
> [junit4] 2> 104071 T443 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
> [junit4] 2> 104146 T443 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 104148 T443 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:40837
> [junit4] 2> 104149 T443 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
> [junit4] 2> 104149 T443 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
> [junit4] 2> 104149 T443 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693
> [junit4] 2> 104150 T443 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/'
> [junit4] 2> 104164 T443 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/solr.xml
> [junit4] 2> 104201 T443 oasc.CoreContainer.<init> New CoreContainer
> 2835059
> [junit4] 2> 104202 T443 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/]
> [junit4] 2> 104203 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 120000
> [junit4] 2> 104203 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to: http://
> [junit4] 2> 104203 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
> [junit4] 2> 104204 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
> [junit4] 2> 104204 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
> [junit4] 2> 104204 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
> [junit4] 2> 104204 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
> [junit4] 2> 104205 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
> [junit4] 2> 104205 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
> [junit4] 2> 104206 T443 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 104206 T443 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 104207 T443 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:58676/solr
> [junit4] 2> 104207 T443 oasc.ZkController.checkChrootPath zkHost
> includes chroot
> [junit4] 2> 104208 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 104210 T499 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@fc606aname:ZooKeeperConnection Watcher:
> 127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 104210 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 104214 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 104215 T501 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@14876b4name:ZooKeeperConnection Watcher:
> 127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 104215 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 104220 T443
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
> [junit4] 2> 105224 T443 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40837_xy_%2Fbz
> [junit4] 2> 105227 T443 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:40837_xy_%2Fbz
> [junit4] 2> 105232 T485 oascc.ZkStateReader$3.process Updating live
> nodes... (3)
> [junit4] 2> 105232 T472 oascc.ZkStateReader$3.process Updating live
> nodes... (3)
> [junit4] 2> 105232 T465 oascc.ZkStateReader$3.process Updating live
> nodes... (3)
> [junit4] 2> 105232 T501 oascc.ZkStateReader$3.process Updating live
> nodes... (3)
> [junit4] 2> 105244 T502 oasc.ZkController.publish publishing
> core=collection1 state=down
> [junit4] 2> 105245 T502 oasc.ZkController.publish numShards not found
> on descriptor - reading it from system property
> [junit4] 2> 105247 T502 oasc.ZkController.waitForCoreNodeName look
> for our core node name
> [junit4] 2> 105530 T466 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 105531 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"http://127.0.0.1:41931/xy_/bz",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:41931_xy_%2Fbz",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 105535 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:40837/xy_/bz",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:40837_xy_%2Fbz",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node2"}
> [junit4] 2> 105535 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Collection already exists with numShards=2
> [junit4] 2> 105536 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Assigning new node to shard shard=shard2
> [junit4] 2> 105539 T485 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 105539 T472 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 105540 T465 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 105540 T501 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 106247 T502 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for collection1
> [junit4] 2> 106248 T502 oasc.CoreContainer.create Creating SolrCore
> 'collection1' using instanceDir:
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1
> [junit4] 2> 106248 T502 oasc.ZkController.createCollectionZkNode
> Check for collection zkNode:collection1
> [junit4] 2> 106250 T502 oasc.ZkController.createCollectionZkNode
> Collection zkNode exists
> [junit4] 2> 106250 T502 oasc.ZkController.readConfigName Load
> collection config from:/collections/collection1
> [junit4] 2> 106252 T502 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1/'
> [junit4] 2> 106253 T502 oasc.SolrResourceLoader.replaceClassLoader
> Adding
> 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1/lib/classes/'
> to classloader
> [junit4] 2> 106254 T502 oasc.SolrResourceLoader.replaceClassLoader
> Adding
> 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1/lib/README'
> to classloader
> [junit4] 2> 106299 T502 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: LUCENE_46
> [junit4] 2> 106337 T502 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
> [junit4] 2> 106338 T502 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
> [junit4] 2> 106346 T502 oass.IndexSchema.readSchema [collection1]
> Schema name=test
> [junit4] 2> 106740 T502 oass.ByteField.init WARN ByteField is
> deprecated and will be removed in 5.0. You should use TrieIntField instead.
> [junit4] 2> 106740 T502 oass.ShortField.init WARN ShortField is
> deprecated and will be removed in 5.0. You should use TrieIntField instead.
> [junit4] 2> 106741 T502 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4] 2> 106746 T502 oass.IndexSchema.readSchema default search
> field in schema is text
> [junit4] 2> 106749 T502 oass.IndexSchema.readSchema unique key field:
> id
> [junit4] 2> 106762 T502 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
> [junit4] 2> 106766 T502 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
> [junit4] 2> 106771 T502 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 106772 T502
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
> [junit4] 2> 106773 T502
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
> [junit4] 2> 106773 T502 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 106774 T502
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
> [junit4] 2> 106774 T502
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
> [junit4] 2> 106775 T502 oasc.SolrCore.initDirectoryFactory
> solr.StandardDirectoryFactory
> [junit4] 2> 106775 T502 oasc.SolrCore.<init> [collection1] Opening
> new SolrCore at
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1382223990693/collection1/,
> dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/
> [junit4] 2> 106775 T502 oasc.JmxMonitoredMap.<init> JMX monitoring is
> enabled. Adding Solr mbeans to JMX Server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@be73e9
> [junit4] 2> 106776 T502 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2
> [junit4] 2> 106777 T502 oasc.SolrCore.getNewIndexDir New index
> directory detected: old=null
> new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index/
> [junit4] 2> 106777 T502 oasc.SolrCore.initIndex WARN [collection1]
> Solr index directory
> './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index'
> doesn't exist. Creating new index...
> [junit4] 2> 106777 T502 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index
> [junit4] 2> 106778 T502 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=29,
> maxMergedSegmentMB=42.689453125, floorSegmentMB=0.5654296875,
> forceMergeDeletesPctAllowed=3.6976139800581977, segmentsPerTier=26.0,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
> [junit4] 2> 106781 T502 oasc.SolrDeletionPolicy.onCommit
> SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2>
> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index,segFN=segments_1,generation=1}
> [junit4] 2> 106781 T502 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
> [junit4] 2> 106783 T502 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 106783 T502 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 106784 T502 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "dedupe"
> [junit4] 2> 106784 T502 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 106784 T502 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "stored_sig"
> [junit4] 2> 106784 T502 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 106785 T502 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 106785 T502 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 106785 T502 oasc.SolrCore.loadUpdateProcessorChains no
> updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 106786 T502 oasc.RequestHandlers.initHandlersFromConfig
> created standard: solr.StandardRequestHandler
> [junit4] 2> 106786 T502 oasc.RequestHandlers.initHandlersFromConfig
> created /get: solr.RealTimeGetHandler
> [junit4] 2> 106787 T502 oasc.RequestHandlers.initHandlersFromConfig
> adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 106787 T502 oasc.RequestHandlers.initHandlersFromConfig
> created /replication: solr.ReplicationHandler
> [junit4] 2> 106787 T502 oasc.RequestHandlers.initHandlersFromConfig
> created /update: solr.UpdateRequestHandler
> [junit4] 2> 106788 T502 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 106790 T502 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
> [junit4] 2> 106792 T502 oasu.CommitTracker.<init> Hard AutoCommit:
> disabled
> [junit4] 2> 106792 T502 oasu.CommitTracker.<init> Soft AutoCommit:
> disabled
> [junit4] 2> 106793 T502 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class
> org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy:
> minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648,
> maxMergeSizeForForcedMerge=9223372036854775807,
> calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
> [junit4] 2> 106794 T502 oasc.SolrDeletionPolicy.onInit
> SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2>
> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty2/index,segFN=segments_1,generation=1}
> [junit4] 2> 106794 T502 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
> [junit4] 2> 106794 T502 oass.SolrIndexSearcher.<init> Opening
> Searcher@191b873 main
> [junit4] 2> 106798 T503 oasc.SolrCore.registerSearcher [collection1]
> Registered new searcher Searcher@191b873main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 106801 T502 oasc.CoreContainer.registerCore registering
> core: collection1
> [junit4] 2> 106801 T502 oasc.ZkController.register Register replica -
> core:collection1 address:http://127.0.0.1:40837/xy_/bzcollection:collection1 shard:shard2
> [junit4] 2> 106803 T502 oascc.SolrZkClient.makePath makePath:
> /collections/collection1/leader_elect/shard2/election
> [junit4] 2> 106808 T502
> oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process
> for shard shard2
> [junit4] 2> 106809 T502
> oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas
> found to continue.
> [junit4] 2> 106810 T502
> oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader -
> try and sync
> [junit4] 2> 106810 T502 oasc.SyncStrategy.sync Sync replicas to
> http://127.0.0.1:40837/xy_/bz/collection1/
> [junit4] 2> 106810 T502 oasc.SyncStrategy.syncReplicas Sync Success -
> now sync replicas to me
> [junit4] 2> 106810 T502 oasc.SyncStrategy.syncToMe
> http://127.0.0.1:40837/xy_/bz/collection1/ has no replicas
> [junit4] 2> 106810 T502
> oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader:
> http://127.0.0.1:40837/xy_/bz/collection1/ shard2
> [junit4] 2> 106811 T502 oascc.SolrZkClient.makePath makePath:
> /collections/collection1/leaders/shard2
> [junit4] 2> 107047 T466 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 107054 T485 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 107054 T501 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 107054 T472 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 107055 T465 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 107066 T502 oasc.ZkController.register We are
> http://127.0.0.1:40837/xy_/bz/collection1/ and leader is
> http://127.0.0.1:40837/xy_/bz/collection1/
> [junit4] 2> 107066 T502 oasc.ZkController.register No LogReplay
> needed for core=collection1 baseURL=http://127.0.0.1:40837/xy_/bz
> [junit4] 2> 107067 T502 oasc.ZkController.checkRecovery I am the
> leader, no recovery necessary
> [junit4] 2> 107067 T502 oasc.ZkController.publish publishing
> core=collection1 state=active
> [junit4] 2> 107067 T502 oasc.ZkController.publish numShards not found
> on descriptor - reading it from system property
> [junit4] 2> 107068 T502 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 107069 T443 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
> [junit4] 2> 107069 T443 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
> [junit4] 2> 107132 T443 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 107134 T443 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:35331
> [junit4] 2> 107135 T443 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
> [junit4] 2> 107135 T443 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
> [junit4] 2> 107135 T443 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690
> [junit4] 2> 107135 T443 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/'
> [junit4] 2> 107146 T443 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/solr.xml
> [junit4] 2> 107184 T443 oasc.CoreContainer.<init> New CoreContainer
> 7674880
> [junit4] 2> 107184 T443 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/]
> [junit4] 2> 107185 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 120000
> [junit4] 2> 107186 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to: http://
> [junit4] 2> 107186 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
> [junit4] 2> 107186 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
> [junit4] 2> 107186 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
> [junit4] 2> 107187 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
> [junit4] 2> 107187 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
> [junit4] 2> 107187 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
> [junit4] 2> 107187 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
> [junit4] 2> 107189 T443 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 107189 T443 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 107189 T443 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:58676/solr
> [junit4] 2> 107190 T443 oasc.ZkController.checkChrootPath zkHost
> includes chroot
> [junit4] 2> 107190 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 107265 T515 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@1e5c27bname:ZooKeeperConnection Watcher:
> 127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 107266 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 107268 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 107269 T517 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@894d7dname:ZooKeeperConnection Watcher:
> 127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 107270 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 107274 T443
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
> [junit4] 2> 108277 T443 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35331_xy_%2Fbz
> [junit4] 2> 108279 T443 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:35331_xy_%2Fbz
> [junit4] 2> 108281 T485 oascc.ZkStateReader$3.process Updating live
> nodes... (4)
> [junit4] 2> 108282 T501 oascc.ZkStateReader$3.process Updating live
> nodes... (4)
> [junit4] 2> 108282 T465 oascc.ZkStateReader$3.process Updating live
> nodes... (4)
> [junit4] 2> 108281 T472 oascc.ZkStateReader$3.process Updating live
> nodes... (4)
> [junit4] 2> 108282 T517 oascc.ZkStateReader$3.process Updating live
> nodes... (4)
> [junit4] 2> 108293 T518 oasc.ZkController.publish publishing
> core=collection1 state=down
> [junit4] 2> 108293 T518 oasc.ZkController.publish numShards not found
> on descriptor - reading it from system property
> [junit4] 2> 108295 T518 oasc.ZkController.waitForCoreNodeName look
> for our core node name
> [junit4] 2> 108559 T466 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 108560 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"http://127.0.0.1:40837/xy_/bz",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:40837_xy_%2Fbz",
> [junit4] 2> "shard":"shard2",
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node2"}
> [junit4] 2> 108564 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:35331/xy_/bz",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:35331_xy_%2Fbz",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node3"}
> [junit4] 2> 108564 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Collection already exists with numShards=2
> [junit4] 2> 108564 T466 oasc.Overseer$ClusterStateUpdater.updateState
> Assigning new node to shard shard=shard1
> [junit4] 2> 108567 T485 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 108567 T465 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 108567 T517 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 108567 T472 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 108567 T501 oascc.ZkStateReader$2.process A cluster state
> change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 109296 T518 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for collection1
> [junit4] 2> 109296 T518 oasc.CoreContainer.create Creating SolrCore
> 'collection1' using instanceDir:
> ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1
> [junit4] 2> 109296 T518 oasc.ZkController.createCollectionZkNode
> Check for collection zkNode:collection1
> [junit4] 2> 109297 T518 oasc.ZkController.createCollectionZkNode
> Collection zkNode exists
> [junit4] 2> 109297 T518 oasc.ZkController.readConfigName Load
> collection config from:/collections/collection1
> [junit4] 2> 109298 T518 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1/'
> [junit4] 2> 109299 T518 oasc.SolrResourceLoader.replaceClassLoader
> Adding
> 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1/lib/classes/'
> to classloader
> [junit4] 2> 109300 T518 oasc.SolrResourceLoader.replaceClassLoader
> Adding
> 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1/lib/README'
> to classloader
> [junit4] 2> 109341 T518 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: LUCENE_46
> [junit4] 2> 109388 T518 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
> [junit4] 2> 109389 T518 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
> [junit4] 2> 109396 T518 oass.IndexSchema.readSchema [collection1]
> Schema name=test
> [junit4] 2> 109783 T518 oass.ByteField.init WARN ByteField is
> deprecated and will be removed in 5.0. You should use TrieIntField instead.
> [junit4] 2> 109784 T518 oass.ShortField.init WARN ShortField is
> deprecated and will be removed in 5.0. You should use TrieIntField instead.
> [junit4] 2> 109785 T518 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4] 2> 109791 T518 oass.IndexSchema.readSchema default search
> field in schema is text
> [junit4] 2> 109793 T518 oass.IndexSchema.readSchema unique key field:
> id
> [junit4] 2> 109804 T518 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
> [junit4] 2> 109807 T518 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
> [junit4] 2> 109810 T518 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 109811 T518
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
> [junit4] 2> 109811 T518
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
> [junit4] 2> 109811 T518 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
> [junit4] 2> 109812 T518
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
> [junit4] 2> 109812 T518
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
> [junit4] 2> 109813 T518 oasc.SolrCore.initDirectoryFactory
> solr.StandardDirectoryFactory
> [junit4] 2> 109813 T518 oasc.SolrCore.<init> [collection1] Opening
> new SolrCore at
> ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1382223993690/collection1/,
> dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/
> [junit4] 2> 109813 T518 oasc.JmxMonitoredMap.<init> JMX monitoring is
> enabled. Adding Solr mbeans to JMX Server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@be73e9
> [junit4] 2> 109814 T518 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3
> [junit4] 2> 109814 T518 oasc.SolrCore.getNewIndexDir New index
> directory detected: old=null
> new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index/
> [junit4] 2> 109814 T518 oasc.SolrCore.initIndex WARN [collection1]
> Solr index directory
> './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index'
> doesn't exist. Creating new index...
> [junit4] 2> 109814 T518 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index
> [junit4] 2> 109815 T518 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=29,
> maxMergedSegmentMB=42.689453125, floorSegmentMB=0.5654296875,
> forceMergeDeletesPctAllowed=3.6976139800581977, segmentsPerTier=26.0,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
> [junit4] 2> 109817 T518 oasc.SolrDeletionPolicy.onCommit
> SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2>
> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index,segFN=segments_1,generation=1}
> [junit4] 2> 109817 T518 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
> [junit4] 2> 109819 T518 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 109820 T518 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 109820 T518 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "dedupe"
> [junit4] 2> 109820 T518 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 109820 T518 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "stored_sig"
> [junit4] 2> 109821 T518 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 109821 T518 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 109821 T518 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 109821 T518 oasc.SolrCore.loadUpdateProcessorChains no
> updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 109822 T518 oasc.RequestHandlers.initHandlersFromConfig
> created standard: solr.StandardRequestHandler
> [junit4] 2> 109823 T518 oasc.RequestHandlers.initHandlersFromConfig
> created /get: solr.RealTimeGetHandler
> [junit4] 2> 109823 T518 oasc.RequestHandlers.initHandlersFromConfig
> adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 109823 T518 oasc.RequestHandlers.initHandlersFromConfig
> created /replication: solr.ReplicationHandler
> [junit4] 2> 109824 T518 oasc.RequestHandlers.initHandlersFromConfig
> created /update: solr.UpdateRequestHandler
> [junit4] 2> 109824 T518 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 109826 T518 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
> [junit4] 2> 109828 T518 oasu.CommitTracker.<init> Hard AutoCommit:
> disabled
> [junit4] 2> 109829 T518 oasu.CommitTracker.<init> Soft AutoCommit:
> disabled
> [junit4] 2> 109829 T518 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class
> org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy:
> minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648,
> maxMergeSizeForForcedMerge=9223372036854775807,
> calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
> [junit4] 2> 109830 T518 oasc.SolrDeletionPolicy.onInit
> SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2>
> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1382223984051/jetty3/index,segFN=segments_1,generation=1}
> [junit4] 2> 109830 T518 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
> [junit4] 2> 109831 T518 oass.SolrIndexSearcher.<init> Opening
> Searcher@14234b0 main
> [junit4] 2> 109835 T519 oasc.SolrCore.registerSearcher [collection1]
> Registered new searcher Searcher@14234b0main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 109837 T518 oasc.CoreContainer.registerCore registering
> core: collection1
> [junit4] 2> 109837 T518 oasc.ZkController.register Register replica -
> core:collection1 address:http://127.0.0.1:35331/xy_/bzcollection:collection1 shard:shard1
> [junit4] 2> 109840 T518 oasc.ZkController.register We are
> http://127.0.0.1:35331/xy_/bz/collection1/ and leader is
> http://127.0.0.1:41931/xy_/bz/collection1/
> [junit4] 2> 109841 T518 oasc.ZkController.register No LogReplay
> needed for core=collection1 baseURL=http://127.0.0.1:35331/xy_/bz
> [junit4] 2> 109841 T518 oasc.ZkController.checkRecovery Core needs to
> recover:collection1
> [junit4] 2> 109841 T518 oasu.DefaultSolrCoreState.doRecovery Running
> recovery - first canceling any ongoing recovery
> [junit4] 2> ASYNC NEW_CORE C291 name=collection1
> org.apache.solr.core.SolrCore@9a575b url=
> http://127.0.0.1:35331/xy_/bz/collection1 node=127.0.0.1:35331_xy_%2Fbz
> C291_STATE=coll:collection1 core:collection1 props:{state=down, base_url=
> http://127.0.0.1:35331/xy_/bz, core=collection1, node_name=127.0.0.1:35331
> _xy_%2Fbz}
> [junit4] 2> 109844 T520 C291 P35331 oasc.RecoveryStrategy.run
> Starting recovery process. core=collection1 recoveringAfterStartup=true
> [junit4] 2> 109844 T518 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
> [junit4] 2> 109845 T520 C291 P35331 oasc.RecoveryStrategy.doRecovery
> ###### startupVersions=[]
> [junit4] 2> 109845 T520 C291 P35331 oasc.ZkController.publish
> publishing core=collection1 state=recovering
> [junit4] 2> 109845 T520 C291 P35331 oasc.ZkController.publish
> numShards not found on descriptor - reading it from system property
> [junit4] 2> 109846 T443 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
> [junit4] 2> 109846 T443 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
> [junit4] 2> 109919 T443 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 109921 T443 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:52309
> [junit4] 2> 109921 T443 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
> [junit4] 2> 109922 T443 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
> [junit4] 2> 109922 T443 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1382223996467
> [junit4] 2> 109923 T443 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1382223996467/'
> [junit4] 2> 109941 T443 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1382223996467/solr.xml
> [junit4] 2> 109990 T443 oasc.CoreContainer.<init> New CoreContainer
> 23229910
> [junit4] 2> 109991 T443 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty4-1382223996467/]
> [junit4] 2> 109992 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 120000
> [junit4] 2> 109992 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to: http://
> [junit4] 2> 109992 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
> [junit4] 2> 109993 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
> [junit4] 2> 109993 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
> [junit4] 2> 109993 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
> [junit4] 2> 109994 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
> [junit4] 2> 109994 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
> [junit4] 2> 109994 T443 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
> [junit4] 2> 109995 T443 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 109996 T443 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 109996 T443 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:58676/solr
> [junit4] 2> 109996 T443 oasc.ZkController.checkChrootPath zkHost
> includes chroot
> [junit4] 2> 109997 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 109998 T532 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@12f23a2name:ZooKeeperConnection Watcher:
> 127.0.0.1:58676 got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 109999 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4] 2> 110001 T443 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
> [junit4] 2> 110002 T534 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@12ac54name:ZooKeeperConnection Watcher:
> 127.0.0.1:58676/solr got event WatchedEvent state:SyncConnected type:None
> path:null path:null type:None
> [junit4] 2> 110002 T443 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
> [junit4]
>
> [...truncated too long message...]
>
> 44_xy_%2Fbz"}}}},
> [junit4] 1> "maxShardsPerNode":"1",
> [junit4] 1> "router":{"name":"compositeId"},
> [junit4] 1> "replicationFactor":"1"},
> [junit4] 1> "control_collection":{
> [junit4] 1> "shards":{"shard1":{
> [junit4] 1> "range":"80000000-7fffffff",
> [junit4] 1> "state":"active",
> [junit4] 1> "replicas":{"core_node1":{
> [junit4] 1> "state":"active",
> [junit4] 1> "base_url":"
> http://127.0.0.1:41444/xy_/bz",
> [junit4] 1> "core":"collection1",
> [junit4] 1> "node_name":"127.0.0.1:41444_xy_%2Fbz",
> [junit4] 1> "leader":"true"}}}},
> [junit4] 1> "maxShardsPerNode":"1",
> [junit4] 1> "router":{"name":"compositeId"},
> [junit4] 1> "replicationFactor":"1"}}
> [junit4] 1> /solr/aliases.json (0)
> [junit4] 1> /solr/live_nodes (8)
> [junit4] 1> /solr/live_nodes/127.0.0.1:40837_xy_%2Fbz (0)
> [junit4] 1> /solr/live_nodes/127.0.0.1:41931_xy_%2Fbz (0)
> [junit4] 1> /solr/live_nodes/127.0.0.1:41444_xy_%2Fbz (0)
> [junit4] 1> /solr/live_nodes/127.0.0.1:36944_xy_%2Fbz (0)
> [junit4] 1> /solr/live_nodes/127.0.0.1:35318_xy_%2Fbz (0)
> [junit4] 1> /solr/live_nodes/127.0.0.1:35331_xy_%2Fbz (0)
> [junit4] 1> /solr/live_nodes/127.0.0.1:52309_xy_%2Fbz (0)
> [junit4] 1> /solr/live_nodes/127.0.0.1:34262_xy_%2Fbz (0)
> [junit4] 1> /solr/overseer (3)
> [junit4] 1> DATA:
> [junit4] 1>
> [junit4] 1> /solr/overseer/queue (0)
> [junit4] 1> /solr/overseer/queue-work (0)
> [junit4] 1> /solr/overseer/collection-queue-work (0)
> [junit4] 1> /solr/collections (2)
> [junit4] 1> /solr/collections/collection1 (3)
> [junit4] 1> DATA:
> [junit4] 1> {"configName":"conf1"}
> [junit4] 1> /solr/collections/collection1/shards (0)
> [junit4] 1> /solr/collections/collection1/leader_elect (2)
> [junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
> [junit4] 1>
> /solr/collections/collection1/leader_elect/shard1/election (4)
> [junit4] 1>
> /solr/collections/collection1/leader_elect/shard1/election/90585431024205838-core_node5-n_0000000002
> (0)
> [junit4] 1>
> /solr/collections/collection1/leader_elect/shard1/election/90585431024205842-core_node7-n_0000000003
> (0)
> [junit4] 1>
> /solr/collections/collection1/leader_elect/shard1/election/90585431024205834-core_node3-n_0000000001
> (0)
> [junit4] 1>
> /solr/collections/collection1/leader_elect/shard1/election/90585431024205830-core_node1-n_0000000000
> (0)
> [junit4] 1> /solr/collections/collection1/leader_elect/shard2 (1)
> [junit4] 1>
> /solr/collections/collection1/leader_elect/shard2/election (3)
> [junit4] 1>
> /solr/collections/collection1/leader_elect/shard2/election/90585431024205840-core_node6-n_0000000002
> (0)
> [junit4] 1>
> /solr/collections/collection1/leader_elect/shard2/election/90585431024205836-core_node4-n_0000000001
> (0)
> [junit4] 1>
> /solr/collections/collection1/leader_elect/shard2/election/90585431024205832-core_node2-n_0000000000
> (0)
> [junit4] 1> /solr/collections/collection1/leaders (2)
> [junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
> [junit4] 1> DATA:
> [junit4] 1> {
> [junit4] 1> "core":"collection1",
> [junit4] 1> "node_name":"127.0.0.1:41931_xy_%2Fbz",
> [junit4] 1> "base_url":"http://127.0.0.1:41931/xy_/bz"}
> [junit4] 1> /solr/collections/collection1/leaders/shard2 (0)
> [junit4] 1> DATA:
> [junit4] 1> {
> [junit4] 1> "core":"collection1",
> [junit4] 1> "node_name":"127.0.0.1:40837_xy_%2Fbz",
> [junit4] 1> "base_url":"http://127.0.0.1:40837/xy_/bz"}
> [junit4] 1> /solr/collections/control_collection (3)
> [junit4] 1> DATA:
> [junit4] 1> {"configName":"conf1"}
> [junit4] 1> /solr/collections/control_collection/shards (0)
> [junit4] 1> /solr/collections/control_collection/leader_elect (1)
> [junit4] 1>
> /solr/collections/control_collection/leader_elect/shard1 (1)
> [junit4] 1>
> /solr/collections/control_collection/leader_elect/shard1/election (1)
> [junit4] 1>
> /solr/collections/control_collection/leader_elect/shard1/election/90585431024205827-core_node1-n_0000000000
> (0)
> [junit4] 1> /solr/collections/control_collection/leaders (1)
> [junit4] 1> /solr/collections/control_collection/leaders/shard1
> (0)
> [junit4] 1> DATA:
> [junit4] 1> {
> [junit4] 1> "core":"collection1",
> [junit4] 1> "node_name":"127.0.0.1:41444_xy_%2Fbz",
> [junit4] 1> "base_url":"http://127.0.0.1:41444/xy_/bz"}
> [junit4] 1> /solr/overseer_elect (2)
> [junit4] 1> /solr/overseer_elect/election (8)
> [junit4] 1>
> /solr/overseer_elect/election/90585431024205838-127.0.0.1:35318_xy_%2Fbz-n_0000000005
> (0)
> [junit4] 1>
> /solr/overseer_elect/election/90585431024205842-127.0.0.1:34262_xy_%2Fbz-n_0000000007
> (0)
> [junit4] 1>
> /solr/overseer_elect/election/90585431024205827-127.0.0.1:41444_xy_%2Fbz-n_0000000000
> (0)
> [junit4] 1>
> /solr/overseer_elect/election/90585431024205840-127.0.0.1:36944_xy_%2Fbz-n_0000000006
> (0)
> [junit4] 1>
> /solr/overseer_elect/election/90585431024205830-127.0.0.1:41931_xy_%2Fbz-n_0000000001
> (0)
> [junit4] 1>
> /solr/overseer_elect/election/90585431024205834-127.0.0.1:35331_xy_%2Fbz-n_0000000003
> (0)
> [junit4] 1>
> /solr/overseer_elect/election/90585431024205836-127.0.0.1:52309_xy_%2Fbz-n_0000000004
> (0)
> [junit4] 1>
> /solr/overseer_elect/election/90585431024205832-127.0.0.1:40837_xy_%2Fbz-n_0000000002
> (0)
> [junit4] 1> /solr/overseer_elect/leader (0)
> [junit4] 1> DATA:
> [junit4] 1>
> {"id":"90585431024205827-127.0.0.1:41444_xy_%2Fbz-n_0000000000"}
> [junit4] 1> /zookeeper (1)
> [junit4] 1> DATA:
> [junit4] 1>
> [junit4] 1>
> [junit4] 2> 156498 T569 oascc.ZkStateReader$3.process WARN ZooKeeper
> watch triggered, but Solr cannot talk to ZK
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.method=testDistribSearch
> -Dtests.seed=F04698EAC795B237 -Dtests.multiplier=3 -Dtests.slow=true
> -Dtests.locale=ar_LY -Dtests.timezone=America/Argentina/San_Juan
> -Dtests.file.encoding=UTF-8
> [junit4] FAILURE 59.1s J1 |
> ChaosMonkeyNothingIsSafeTest.testDistribSearch <<<
> [junit4] > Throwable #1: java.lang.AssertionError: shard1 is not
> consistent. Got 103 from
> http://127.0.0.1:35318/xy_/bz/collection1lastClient and got 102 from
> http://127.0.0.1:34262/xy_/bz/collection1
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([F04698EAC795B237:71A016F2B0CAD20B]:0)
> [junit4] > at
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1130)
> [junit4] > at
> org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:194)
> [junit4] > at
> org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
> [junit4] > at java.lang.Thread.run(Thread.java:744)
> [junit4] 2> 156516 T443 oas.SolrTestCaseJ4.deleteCore ###deleteCore
> [junit4] 2> 59102 T442 ccr.ThreadLeakControl.checkThreadLeaks WARNING
> Will linger awaiting termination of 1 leaked thread(s).
> [junit4] 2> 157622 T588 oascc.ZkStateReader$3.process WARN ZooKeeper
> watch triggered, but Solr cannot talk to ZK
> [junit4] 2> NOTE: test params are: codec=Appending,
> sim=DefaultSimilarity, locale=ar_LY, timezone=America/Argentina/San_Juan
> [junit4] 2> NOTE: Linux 3.8.0-31-generic i386/Oracle Corporation
> 1.7.0_45 (32-bit)/cpus=8,threads=1,free=105194416,total=205258752
> [junit4] 2> NOTE: All tests run in this JVM: [TestSolrXmlPersistence,
> TestFieldTypeResource, SuggesterWFSTTest, TestDFRSimilarityFactory,
> XsltUpdateRequestHandlerTest, RequestHandlersTest,
> TestManagedSchemaFieldResource, TestIndexSearcher,
> TestSolrQueryParserResource, TestStressLucene, SimplePostToolTest,
> TestBinaryResponseWriter, TestInfoStreamLogging, TestNumberUtils,
> TestCollationField, TestIBSimilarityFactory,
> OpenExchangeRatesOrgProviderTest, TestAtomicUpdateErrorCases,
> DirectSolrConnectionTest, PreAnalyzedUpdateProcessorTest,
> SpellCheckComponentTest, TestStressReorder, TestStressRecovery,
> NumericFieldsTest, ChaosMonkeyNothingIsSafeTest]
> [junit4] Completed on J1 in 60.39s, 1 test, 1 failure <<< FAILURES!
>
> [...truncated 825 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:428: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:408: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:483:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1263:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:906:
> There were test failures: 327 suites, 1453 tests, 1 failure, 35 ignored (6
> assumptions)
>
> Total time: 46 minutes 29 seconds
> Build step 'Invoke Ant' marked build as failure
> Description set: Java: 32bit/jdk1.7.0_45 -client -XX:+UseParallelGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
--
- Mark