You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/02/25 17:22:23 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60-ea-b04) -
Build # 9593 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9593/
Java: 32bit/jdk1.7.0_60-ea-b04 -server -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testShutdown
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:46715 within 30000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:46715 within 30000 ms
at __randomizedtesting.SeedInfo.seed([24FF72662020E238:C7897BF3C75A774A]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:200)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:46715 within 30000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:222)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
... 47 more
Build Log:
[...truncated 11643 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 30910 T129 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 30911 T129 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 30912 T129 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 30916 T129 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1393345234799
[junit4] 2> 30919 T129 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 30921 T130 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 31020 T129 oasc.ZkTestServer.run start zk server on port:46715
[junit4] 2> 31056 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 69518 T133 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 38447ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 69519 T129 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 69531 T131 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14469d907a50000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 69532 T129 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46715 46715
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testShutdown -Dtests.seed=24FF72662020E238 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_JO -Dtests.timezone=America/Argentina/Buenos_Aires -Dtests.file.encoding=UTF-8
[junit4] ERROR 38.9s J0 | CloudSolrServerTest.testShutdown <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:46715 within 30000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([24FF72662020E238:C7897BF3C75A774A]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:200)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:77)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:46715 within 30000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:222)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
[junit4] > ... 47 more
[junit4] 2> 69809 T129 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1393345273692
[junit4] 2> 69810 T129 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 69811 T137 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 69911 T129 oasc.ZkTestServer.run start zk server on port:50142
[junit4] 2> 69912 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 69949 T143 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ef393c name:ZooKeeperConnection Watcher:127.0.0.1:50142 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 69949 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 69950 T129 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 69969 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 69971 T145 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1872a61 name:ZooKeeperConnection Watcher:127.0.0.1:50142/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 69971 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 69977 T129 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 69987 T129 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 69992 T129 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 69996 T129 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 70000 T129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 70011 T129 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 70017 T129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 70018 T129 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 70022 T129 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 70023 T129 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 70023 T129 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 70024 T129 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 70024 T129 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 70025 T129 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 70025 T129 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 70026 T129 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 70026 T129 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 70031 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 70033 T147 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15cb827 name:ZooKeeperConnection Watcher:127.0.0.1:50142/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 70033 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 70079 T129 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 70083 T129 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 70099 T129 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:54791
[junit4] 2> 70101 T129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 70102 T129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 70102 T129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393345273920
[junit4] 2> 70103 T129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393345273920/'
[junit4] 2> 70120 T129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393345273920/solr.xml
[junit4] 2> 70168 T129 oasc.CoreContainer.<init> New CoreContainer 19222576
[junit4] 2> 70168 T129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393345273920/]
[junit4] 2> 70170 T129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 70170 T129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 70171 T129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 70171 T129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 70171 T129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 70172 T129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 70172 T129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 70173 T129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 70173 T129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 70180 T129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 70181 T129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 70181 T129 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 70182 T129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50142/solr
[junit4] 2> 70190 T129 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 70191 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 70193 T158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@184da34 name:ZooKeeperConnection Watcher:127.0.0.1:50142 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 70193 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 70198 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 70200 T160 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18a669 name:ZooKeeperConnection Watcher:127.0.0.1:50142/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 70200 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 70205 T129 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 70211 T129 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 70225 T129 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 70229 T129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54791_
[junit4] 2> 70231 T129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54791_
[junit4] 2> 70236 T129 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 70239 T129 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 70247 T129 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:54791_
[junit4] 2> 70247 T129 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 70250 T129 oasc.Overseer.start Overseer (id=91314275856809988-127.0.0.1:54791_-n_0000000000) starting
[junit4] 2> 70261 T129 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 70274 T162 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 70274 T129 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 70275 T162 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 70278 T129 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 70280 T129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 70290 T161 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 70297 T163 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 70298 T163 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 70300 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 70301 T163 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 70303 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 70305 T161 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:54791",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54791_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 70305 T161 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 70314 T161 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 70318 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 70323 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71301 T163 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 71301 T163 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393345273920/collection1
[junit4] 2> 71302 T163 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 71303 T163 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 71303 T163 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 71304 T163 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 71305 T163 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393345273920/collection1/'
[junit4] 2> 71335 T163 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 71368 T163 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 71370 T163 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 71411 T163 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 71458 T163 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 71465 T163 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableLongField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 71470 T163 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableFloatField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 71477 T163 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableDoubleField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 71769 T163 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 71771 T163 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 71773 T163 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 71773 T163 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 71776 T163 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 71776 T163 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393345273920/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/control/data/
[junit4] 2> 71776 T163 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 71777 T163 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/control/data
[junit4] 2> 71777 T163 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/control/data/index/
[junit4] 2> 71777 T163 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 71778 T163 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/control/data/index
[junit4] 2> 71779 T163 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@141358 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 71779 T163 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 71780 T163 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 71780 T163 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 71780 T163 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 71780 T163 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 71781 T163 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 71781 T163 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 71781 T163 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 71781 T163 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 71783 T163 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 71785 T163 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 71786 T163 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 71786 T163 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@141358 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 71787 T163 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 71787 T163 oass.SolrIndexSearcher.<init> Opening Searcher@1499742[collection1] main
[junit4] 2> 71788 T164 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1499742[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 71788 T163 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 71789 T129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 71790 T129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 71789 T167 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:54791 collection:control_collection shard:shard1
[junit4] 2> 71796 T167 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 71798 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 71800 T169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@142c49a name:ZooKeeperConnection Watcher:127.0.0.1:50142/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 71800 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 71802 T129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 71806 T167 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 71807 T129 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 71808 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71809 T167 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 71809 T167 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C24 name=collection1 org.apache.solr.core.SolrCore@ccfd11 url=https://127.0.0.1:54791/collection1 node=127.0.0.1:54791_ C24_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:54791, core=collection1, node_name=127.0.0.1:54791_}
[junit4] 2> 71810 T167 C24 P54791 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:54791/collection1/
[junit4] 2> 71811 T167 C24 P54791 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 71811 T167 C24 P54791 oasc.SyncStrategy.syncToMe https://127.0.0.1:54791/collection1/ has no replicas
[junit4] 2> 71811 T167 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:54791/collection1/ shard1
[junit4] 2> 71811 T167 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 71813 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 71820 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71839 T129 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 71840 T129 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 71844 T129 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 71846 T129 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:53334
[junit4] 2> 71848 T129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 71848 T129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 71849 T129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393345275689
[junit4] 2> 71849 T129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393345275689/'
[junit4] 2> 71859 T129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393345275689/solr.xml
[junit4] 2> 71890 T129 oasc.CoreContainer.<init> New CoreContainer 8804305
[junit4] 2> 71890 T129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393345275689/]
[junit4] 2> 71891 T129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 71892 T129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 71892 T129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 71892 T129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 71893 T129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 71893 T129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 71893 T129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 71893 T129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 71894 T129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 71899 T129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 71899 T129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 71899 T129 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 71900 T129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50142/solr
[junit4] 2> 71900 T129 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 71902 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 71933 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71933 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 71936 T180 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cdc8e7 name:ZooKeeperConnection Watcher:127.0.0.1:50142 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 71936 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 71940 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 71941 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 71942 T182 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@282d91 name:ZooKeeperConnection Watcher:127.0.0.1:50142/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 71943 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 71946 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 71950 T129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 72049 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 72049 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 72049 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 72087 T167 oasc.ZkController.register We are https://127.0.0.1:54791/collection1/ and leader is https://127.0.0.1:54791/collection1/
[junit4] 2> 72087 T167 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:54791
[junit4] 2> 72087 T167 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 72087 T167 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 72087 T167 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 72089 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72089 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72089 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72090 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 72092 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 72093 T161 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:54791",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54791_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 72099 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72202 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 72202 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 72202 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 72954 T129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53334_
[junit4] 2> 72957 T129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53334_
[junit4] 2> 72961 T169 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 72961 T160 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 72963 T182 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 72972 T183 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 72972 T183 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 72974 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72974 T183 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 72974 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72975 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 72976 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 72977 T161 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:53334",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:53334_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 72977 T161 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 72978 T161 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 72990 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 73094 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 73094 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 73094 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 73975 T183 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 73975 T183 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393345275689/collection1
[junit4] 2> 73975 T183 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 73976 T183 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 73976 T183 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 73978 T183 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 73978 T183 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393345275689/collection1/'
[junit4] 2> 74006 T183 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 74031 T183 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 74033 T183 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 74051 T183 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 74337 T183 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 74340 T183 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 74342 T183 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 74342 T183 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 74343 T183 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 74343 T183 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393345275689/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty1/
[junit4] 2> 74343 T183 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 74344 T183 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty1
[junit4] 2> 74344 T183 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty1/index/
[junit4] 2> 74344 T183 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 74344 T183 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty1/index
[junit4] 2> 74346 T183 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@758ddb lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 74346 T183 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74347 T183 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 74347 T183 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 74347 T183 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 74347 T183 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 74348 T183 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 74348 T183 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 74348 T183 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 74348 T183 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 74350 T183 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 74352 T183 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 74353 T183 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 74353 T183 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@758ddb lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 74354 T183 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74354 T183 oass.SolrIndexSearcher.<init> Opening Searcher@261eab[collection1] main
[junit4] 2> 74355 T184 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@261eab[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 74355 T183 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 74356 T129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 74356 T187 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:53334 collection:collection1 shard:shard1
[junit4] 2> 74356 T129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 74358 T187 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 74379 T187 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 74383 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74383 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74383 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74384 T187 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 74384 T187 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C25 name=collection1 org.apache.solr.core.SolrCore@1afb661 url=https://127.0.0.1:53334/collection1 node=127.0.0.1:53334_ C25_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:53334, core=collection1, node_name=127.0.0.1:53334_}
[junit4] 2> 74385 T187 C25 P53334 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:53334/collection1/
[junit4] 2> 74389 T187 C25 P53334 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 74390 T187 C25 P53334 oasc.SyncStrategy.syncToMe https://127.0.0.1:53334/collection1/ has no replicas
[junit4] 2> 74390 T187 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:53334/collection1/ shard1
[junit4] 2> 74390 T187 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 74393 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 74400 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74408 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74414 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74419 T129 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 74420 T129 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 74425 T129 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 74427 T129 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:43624
[junit4] 2> 74430 T129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 74431 T129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 74431 T129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393345278240
[junit4] 2> 74432 T129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393345278240/'
[junit4] 2> 74443 T129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393345278240/solr.xml
[junit4] 2> 74476 T129 oasc.CoreContainer.<init> New CoreContainer 19478221
[junit4] 2> 74477 T129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393345278240/]
[junit4] 2> 74477 T129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 74478 T129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 74478 T129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 74478 T129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 74479 T129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 74479 T129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 74479 T129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 74480 T129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 74480 T129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 74486 T129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 74486 T129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 74487 T129 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 74487 T129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50142/solr
[junit4] 2> 74488 T129 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 74489 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 74510 T198 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@280c42 name:ZooKeeperConnection Watcher:127.0.0.1:50142 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 74510 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 74532 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 74551 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74551 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74551 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74552 T200 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@284ba3 name:ZooKeeperConnection Watcher:127.0.0.1:50142/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 74552 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 74559 T187 oasc.ZkController.register We are https://127.0.0.1:53334/collection1/ and leader is https://127.0.0.1:53334/collection1/
[junit4] 2> 74560 T187 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:53334
[junit4] 2> 74560 T187 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 74561 T187 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 74561 T187 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 74565 T187 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 74565 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74566 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74570 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 74572 T161 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:53334",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:53334_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 74572 T129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 74577 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74682 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74682 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74682 T200 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 74682 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 75580 T129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43624_
[junit4] 2> 75582 T129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43624_
[junit4] 2> 75586 T182 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 75586 T160 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 75586 T200 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 75586 T169 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 75601 T201 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 75602 T201 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 75603 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75603 T201 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 75603 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75603 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75606 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 75607 T161 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43624",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:43624_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 75607 T161 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 75608 T161 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 75727 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75832 T200 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 75832 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 75832 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 75832 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 76603 T201 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 76604 T201 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393345278240/collection1
[junit4] 2> 76604 T201 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 76605 T201 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 76605 T201 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 76607 T201 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 76607 T201 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393345278240/collection1/'
[junit4] 2> 76633 T201 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 76657 T201 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 76659 T201 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 76676 T201 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 76931 T201 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 76934 T201 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 76936 T201 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 76936 T201 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 76937 T201 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 76937 T201 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393345278240/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty2/
[junit4] 2> 76937 T201 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 76938 T201 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty2
[junit4] 2> 76938 T201 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty2/index/
[junit4] 2> 76938 T201 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 76938 T201 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty2/index
[junit4] 2> 76939 T201 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@83382b lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 76939 T201 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 76940 T201 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 76941 T201 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 76941 T201 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 76941 T201 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 76941 T201 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 76942 T201 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 76942 T201 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 76942 T201 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 76944 T201 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 76948 T201 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 76948 T201 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 76948 T201 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@83382b lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 76949 T201 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 76949 T201 oass.SolrIndexSearcher.<init> Opening Searcher@b5b129[collection1] main
[junit4] 2> 76950 T202 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b5b129[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 76950 T201 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 76951 T129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 76951 T129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 76951 T205 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:43624 collection:collection1 shard:shard2
[junit4] 2> 76953 T205 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 76961 T205 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 76963 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76964 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76964 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76964 T205 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 76964 T205 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C26 name=collection1 org.apache.solr.core.SolrCore@1c7e9d0 url=https://127.0.0.1:43624/collection1 node=127.0.0.1:43624_ C26_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:43624, core=collection1, node_name=127.0.0.1:43624_}
[junit4] 2> 76964 T205 C26 P43624 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:43624/collection1/
[junit4] 2> 76965 T205 C26 P43624 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 76965 T205 C26 P43624 oasc.SyncStrategy.syncToMe https://127.0.0.1:43624/collection1/ has no replicas
[junit4] 2> 76965 T205 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:43624/collection1/ shard2
[junit4] 2> 76965 T205 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 76965 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 76969 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76971 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76975 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76980 T129 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 76980 T129 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 76984 T129 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 76986 T129 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51391
[junit4] 2> 76988 T129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 76988 T129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 76988 T129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393345280835
[junit4] 2> 76989 T129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393345280835/'
[junit4] 2> 76999 T129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393345280835/solr.xml
[junit4] 2> 77027 T129 oasc.CoreContainer.<init> New CoreContainer 22754026
[junit4] 2> 77028 T129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393345280835/]
[junit4] 2> 77028 T129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 77029 T129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 77029 T129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 77029 T129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 77030 T129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 77030 T129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 77030 T129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 77030 T129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 77031 T129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 77036 T129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 77037 T129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 77037 T129 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 77037 T129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50142/solr
[junit4] 2> 77038 T129 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 77039 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77041 T216 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@834b16 name:ZooKeeperConnection Watcher:127.0.0.1:50142 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77042 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77048 T129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77050 T218 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17bb1f5 name:ZooKeeperConnection Watcher:127.0.0.1:50142/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77051 T129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77058 T129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 77079 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77079 T200 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77080 T218 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77080 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77079 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77122 T205 oasc.ZkController.register We are https://127.0.0.1:43624/collection1/ and leader is https://127.0.0.1:43624/collection1/
[junit4] 2> 77123 T205 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:43624
[junit4] 2> 77123 T205 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 77123 T205 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 77123 T205 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 77125 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77126 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77126 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77126 T205 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77128 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77130 T161 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:43624",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:43624_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 77133 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77238 T218 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77238 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77239 T200 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77239 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 77242 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78063 T129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51391_
[junit4] 2> 78064 T129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51391_
[junit4] 2> 78068 T218 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78069 T200 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78069 T169 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78069 T160 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78070 T182 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 78079 T219 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 78079 T219 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 78080 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78080 T219 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 78080 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78081 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78083 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 78084 T161 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:51391",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:51391_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 78085 T161 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 78085 T161 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 78090 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78195 T200 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 78195 T218 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 78195 T182 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 78195 T160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 78195 T169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 79081 T219 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 79081 T219 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393345280835/collection1
[junit4] 2> 79081 T219 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 79082 T219 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 79083 T219 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 79084 T219 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 79084 T219 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393345280835/collection1/'
[junit4] 2> 79110 T219 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 79137 T219 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 79139 T219 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 79152 T219 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 79393 T219 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 79395 T219 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 79398 T219 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 79399 T219 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 79399 T219 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 79399 T219 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393345280835/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3/
[junit4] 2> 79400 T219 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 79400 T219 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3
[junit4] 2> 79400 T219 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3/index/
[junit4] 2> 79400 T219 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 79401 T219 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3/index
[junit4] 2> 79402 T219 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@1a67a20 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3/index),segFN=segments_1,generation=1}
[junit4] 2> 79402 T219 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79403 T219 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 79403 T219 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 79403 T219 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 79404 T219 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 79404 T219 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 79404 T219 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 79404 T219 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 79405 T219 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 79407 T219 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 79409 T219 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 79409 T219 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 79410 T219 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@1a67a20 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3/index),segFN=segments_1,generation=1}
[junit4] 2> 79410 T219 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79410 T219 oass.SolrIndexSearcher.<init> Opening Searcher@e451ad[collection1] main
[junit4] 2> 79411 T220 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e451ad[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 79411 T219 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 79412 T129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 79412 T129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 79412 T223 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:51391 collection:collection1 shard:shard1
[junit4] 2> 79417 T223 oasc.ZkController.register We are https://127.0.0.1:51391/collection1/ and leader is https://127.0.0.1:53334/collection1/
[junit4] 2> 79417 T223 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:51391
[junit4] 2> 79417 T223 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 79417 T223 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C27 name=collection1 org.apache.solr.core.SolrCore@149d568 url=https://127.0.0.1:51391/collection1 node=127.0.0.1:51391_ C27_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:51391, core=collection1, node_name=127.0.0.1:51391_}
[junit4] 2> 79421 T224 C27 P51391 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 79421 T224 C27 P51391 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 79422 T224 C27 P51391 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 79422 T224 C27 P51391 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 79422 T223 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 79423 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79423 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79423 T160 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79426 T161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 79430 T161 oasc
[...truncated too long message...]
ing DirectUpdateHandler2{commits=6,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=2,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=5}
[junit4] 2> 92869 T218 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 92869 T218 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 92869 T218 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 92871 T218 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 92871 T218 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 92871 T218 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3;done=false>>]
[junit4] 2> 92871 T218 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3
[junit4] 2> 92871 T218 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3/index;done=false>>]
[junit4] 2> 92872 T218 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty3/index
[junit4] 2> 92872 T218 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 92872 T218 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 92872 T218 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 92873 T218 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> ASYNC NEW_CORE C47 name=collection1 org.apache.solr.core.SolrCore@a20429 url=https://127.0.0.1:53130/collection1 node=127.0.0.1:53130_ C47_STATE=coll:collection1 core:collection1 props:{state=active, base_url=https://127.0.0.1:53130, core=collection1, node_name=127.0.0.1:53130_}
[junit4] 2> 92970 T237 C47 P53130 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 92970 T237 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 92970 T237 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@a20429
[junit4] 2> 92971 T237 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=6,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=2,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=5}
[junit4] 2> 92971 T237 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 92971 T237 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 92972 T237 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 92974 T237 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 92975 T237 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 92975 T237 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty4;done=false>>]
[junit4] 2> 92975 T237 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty4
[junit4] 2> 92976 T237 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty4/index;done=false>>]
[junit4] 2> 92976 T237 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393345273692/jetty4/index
[junit4] 2> 92977 T237 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 92977 T237 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Lucene46: {a_t=PostingsFormat(name=Asserting), text=FST41, multiDefault=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=FSTOrdPulsing41), range_facet_sl=MockVariableIntBlock(baseBlockSize=75), range_facet_si=PostingsFormat(name=Asserting), timestamp=PostingsFormat(name=Asserting), range_facet_l=PostingsFormat(name=Asserting), intDefault=PostingsFormat(name=FSTOrdPulsing41), id=MockVariableIntBlock(baseBlockSize=75)}, docValues:{}, sim=DefaultSimilarity, locale=ar_JO, timezone=America/Argentina/Buenos_Aires
[junit4] 2> NOTE: Linux 3.8.0-36-generic i386/Oracle Corporation 1.7.0_60-ea (32-bit)/cpus=8,threads=1,free=34230096,total=72728576
[junit4] 2> NOTE: All tests run in this JVM: [ClientUtilsTest, TermsResponseTest, TestEmbeddedSolrServer, SolrExampleStreamingBinaryTest, TestUpdateRequestCodec, JettyWebappTest, AnlysisResponseBaseTest, ContentStreamTest, SolrExceptionTest, TestXMLEscaping, TestDocumentObjectBinder, LargeVolumeBinaryJettyTest, CloudSolrServerTest]
[junit4] Completed on J0 in 63.00s, 2 tests, 1 error <<< FAILURES!
[...truncated 85 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:465: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:445: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1274: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 48 suites, 280 tests, 1 error
Total time: 54 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b04 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0-fcs-b129) - Build
# 9597 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9597/
Java: 64bit/jdk1.8.0-fcs-b129 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch
Error Message:
commitWithin did not work expected:<68> but was:<67>
Stack Trace:
java.lang.AssertionError: commitWithin did not work expected:<68> but was:<67>
at __randomizedtesting.SeedInfo.seed([4CDAA86DE1E3E202:CD3C267596BC823E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:354)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Build Log:
[...truncated 10430 lines...]
[junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4] 2> 643277 T2745 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 643277 T2745 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /pf_uir/o
[junit4] 2> 643280 T2745 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1393378926844
[junit4] 2> 643280 T2745 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 643281 T2746 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 643381 T2745 oasc.ZkTestServer.run start zk server on port:37619
[junit4] 2> 643381 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 643417 T2752 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@466897a7 name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 643417 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 643417 T2745 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 643421 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 643422 T2754 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5546fdef name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 643422 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 643422 T2745 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 643424 T2745 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 643427 T2745 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 643429 T2745 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 643432 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 643432 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 643435 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 643435 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 643438 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 643438 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 643440 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 643440 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 643442 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 643442 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 643444 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 643444 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 643446 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 643446 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 643448 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 643449 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 643450 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 643451 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 643452 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 643453 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 643455 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 643455 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 643659 T2745 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 643661 T2745 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44752
[junit4] 2> 643662 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 643662 T2745 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 643662 T2745 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022
[junit4] 2> 643663 T2745 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/'
[junit4] 2> 643671 T2745 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/solr.xml
[junit4] 2> 643677 T2745 oasc.CoreContainer.<init> New CoreContainer 1495401464
[junit4] 2> 643677 T2745 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/]
[junit4] 2> 643677 T2745 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 643679 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 643679 T2745 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 643679 T2745 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 643685 T2745 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 643685 T2745 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 643686 T2745 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 643686 T2745 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37619/solr
[junit4] 2> 643686 T2745 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 643687 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 643688 T2765 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61a9a631 name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 643688 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 643690 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 643691 T2767 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d934628 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 643691 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 643692 T2745 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 643694 T2745 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 643696 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 643697 T2745 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44752_pf_uir%2Fo
[junit4] 2> 643697 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44752_pf_uir%2Fo
[junit4] 2> 643699 T2745 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 643700 T2745 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 643702 T2745 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:44752_pf_uir%2Fo
[junit4] 2> 643702 T2745 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 643706 T2745 oasc.Overseer.start Overseer (id=91316481349779459-127.0.0.1:44752_pf_uir%2Fo-n_0000000000) starting
[junit4] 2> 643709 T2745 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 643711 T2769 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 643712 T2745 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 643712 T2769 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 643713 T2745 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 643714 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 643715 T2768 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 643716 T2770 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 643716 T2770 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 643718 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 643718 T2770 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 643719 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 643720 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44752/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:44752_pf_uir%2Fo",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 643720 T2768 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 643720 T2768 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 643721 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 643722 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 644718 T2770 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 644718 T2770 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1
[junit4] 2> 644718 T2770 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 644719 T2770 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 644719 T2770 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 644720 T2770 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 644720 T2770 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/'
[junit4] 2> 644721 T2770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/lib/.svn/' to classloader
[junit4] 2> 644721 T2770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/lib/classes/' to classloader
[junit4] 2> 644721 T2770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/lib/README' to classloader
[junit4] 2> 644735 T2770 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 644742 T2770 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 644743 T2770 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 644746 T2770 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 644828 T2770 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 644831 T2770 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 644832 T2770 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 644840 T2770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 644842 T2770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 644844 T2770 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 644846 T2770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 644846 T2770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 644846 T2770 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 644847 T2770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 644847 T2770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 644847 T2770 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 644847 T2770 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data/
[junit4] 2> 644848 T2770 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4a7385
[junit4] 2> 644848 T2770 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data
[junit4] 2> 644848 T2770 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data/index/
[junit4] 2> 644849 T2770 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 644849 T2770 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data/index
[junit4] 2> 644849 T2770 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=2.447265625, floorSegmentMB=1.44921875, forceMergeDeletesPctAllowed=3.3196981008770865, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1240017361539163
[junit4] 2> 644849 T2770 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@53a74e8f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@736a53af),segFN=segments_1,generation=1}
[junit4] 2> 644850 T2770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 644851 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 644851 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 644851 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 644852 T2770 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 644854 T2770 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 644854 T2770 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 644854 T2770 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 644855 T2770 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 644856 T2770 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 644857 T2770 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 644857 T2770 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2564475347007128]
[junit4] 2> 644857 T2770 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@53a74e8f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@736a53af),segFN=segments_1,generation=1}
[junit4] 2> 644857 T2770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 644857 T2770 oass.SolrIndexSearcher.<init> Opening Searcher@4de84a8d[collection1] main
[junit4] 2> 644859 T2771 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4de84a8d[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 644860 T2770 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 644860 T2745 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 644860 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 644860 T2774 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44752/pf_uir/o collection:control_collection shard:shard1
[junit4] 2> 644861 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 644862 T2776 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32ffa0ca name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 644862 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 644863 T2774 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 644863 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 644865 T2745 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 644867 T2774 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 644868 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 644869 T2774 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 644869 T2774 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1053 name=collection1 org.apache.solr.core.SolrCore@4bab5be0 url=http://127.0.0.1:44752/pf_uir/o/collection1 node=127.0.0.1:44752_pf_uir%2Fo C1053_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:44752/pf_uir/o, core=collection1, node_name=127.0.0.1:44752_pf_uir%2Fo}
[junit4] 2> 644869 T2774 C1053 P44752 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44752/pf_uir/o/collection1/
[junit4] 2> 644869 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 644869 T2774 C1053 P44752 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 644869 T2774 C1053 P44752 oasc.SyncStrategy.syncToMe http://127.0.0.1:44752/pf_uir/o/collection1/ has no replicas
[junit4] 2> 644870 T2774 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44752/pf_uir/o/collection1/ shard1
[junit4] 2> 644870 T2774 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 644871 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 644873 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 644878 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 644980 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 644980 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 645024 T2774 oasc.ZkController.register We are http://127.0.0.1:44752/pf_uir/o/collection1/ and leader is http://127.0.0.1:44752/pf_uir/o/collection1/
[junit4] 2> 645024 T2774 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44752/pf_uir/o
[junit4] 2> 645025 T2774 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 645025 T2774 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 645025 T2774 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 645026 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 645026 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 645026 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 645026 T2774 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 645027 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 645028 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:44752/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:44752_pf_uir%2Fo",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 645031 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 645069 T2745 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 645069 T2745 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 645071 T2745 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43774
[junit4] 2> 645072 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 645072 T2745 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 645072 T2745 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429
[junit4] 2> 645072 T2745 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/'
[junit4] 2> 645081 T2745 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/solr.xml
[junit4] 2> 645092 T2745 oasc.CoreContainer.<init> New CoreContainer 1388692918
[junit4] 2> 645092 T2745 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/]
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 645094 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 645094 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 645094 T2745 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 645094 T2745 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 645096 T2745 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 645096 T2745 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 645096 T2745 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 645096 T2745 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37619/solr
[junit4] 2> 645097 T2745 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 645097 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 645098 T2787 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40a0ca8c name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 645099 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 645100 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 645101 T2789 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74458bb8 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 645101 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 645103 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 645132 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 645132 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 645132 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 646106 T2745 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43774_pf_uir%2Fo
[junit4] 2> 646107 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43774_pf_uir%2Fo
[junit4] 2> 646108 T2776 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 646108 T2789 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 646108 T2767 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 646112 T2790 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 646112 T2790 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 646112 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 646112 T2790 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 646112 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 646113 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 646113 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 646113 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43774/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:43774_pf_uir%2Fo",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 646114 T2768 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 646114 T2768 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 646115 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 646220 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 646220 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 646220 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647113 T2790 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 647113 T2790 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1
[junit4] 2> 647113 T2790 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 647114 T2790 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 647114 T2790 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 647115 T2790 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 647115 T2790 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/'
[junit4] 2> 647115 T2790 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/lib/.svn/' to classloader
[junit4] 2> 647116 T2790 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/lib/classes/' to classloader
[junit4] 2> 647116 T2790 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/lib/README' to classloader
[junit4] 2> 647130 T2790 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 647137 T2790 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 647138 T2790 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 647142 T2790 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 647238 T2790 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 647241 T2790 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 647242 T2790 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 647250 T2790 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 647252 T2790 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 647253 T2790 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 647254 T2790 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 647254 T2790 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 647254 T2790 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 647255 T2790 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 647255 T2790 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 647255 T2790 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 647255 T2790 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1/
[junit4] 2> 647255 T2790 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4a7385
[junit4] 2> 647256 T2790 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1
[junit4] 2> 647256 T2790 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1/index/
[junit4] 2> 647256 T2790 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 647256 T2790 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1/index
[junit4] 2> 647257 T2790 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=2.447265625, floorSegmentMB=1.44921875, forceMergeDeletesPctAllowed=3.3196981008770865, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1240017361539163
[junit4] 2> 647257 T2790 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@46c55a5e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44bcea69),segFN=segments_1,generation=1}
[junit4] 2> 647257 T2790 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 647259 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 647259 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 647259 T2790 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 647260 T2790 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 647261 T2790 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 647261 T2790 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 647261 T2790 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 647261 T2790 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 647262 T2790 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 647262 T2790 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 647262 T2790 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 647262 T2790 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 647264 T2790 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 647265 T2790 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 647265 T2790 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 647265 T2790 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2564475347007128]
[junit4] 2> 647265 T2790 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@46c55a5e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44bcea69),segFN=segments_1,generation=1}
[junit4] 2> 647266 T2790 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 647266 T2790 oass.SolrIndexSearcher.<init> Opening Searcher@7bc891fa[collection1] main
[junit4] 2> 647267 T2791 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7bc891fa[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 647268 T2790 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 647269 T2745 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 647269 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 647269 T2794 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43774/pf_uir/o collection:collection1 shard:shard2
[junit4] 2> 647270 T2794 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 647274 T2794 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 647275 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647275 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647275 T2794 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 647275 T2794 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 647275 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C1054 name=collection1 org.apache.solr.core.SolrCore@c915954 url=http://127.0.0.1:43774/pf_uir/o/collection1 node=127.0.0.1:43774_pf_uir%2Fo C1054_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:43774/pf_uir/o, core=collection1, node_name=127.0.0.1:43774_pf_uir%2Fo}
[junit4] 2> 647275 T2794 C1054 P43774 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43774/pf_uir/o/collection1/
[junit4] 2> 647276 T2794 C1054 P43774 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 647276 T2794 C1054 P43774 oasc.SyncStrategy.syncToMe http://127.0.0.1:43774/pf_uir/o/collection1/ has no replicas
[junit4] 2> 647276 T2794 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43774/pf_uir/o/collection1/ shard2
[junit4] 2> 647276 T2794 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 647276 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 647279 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647280 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647282 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647384 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647384 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647384 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647431 T2794 oasc.ZkController.register We are http://127.0.0.1:43774/pf_uir/o/collection1/ and leader is http://127.0.0.1:43774/pf_uir/o/collection1/
[junit4] 2> 647431 T2794 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43774/pf_uir/o
[junit4] 2> 647431 T2794 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 647431 T2794 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 647431 T2794 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 647432 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647432 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647433 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647433 T2794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 647433 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 647434 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:43774/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:43774_pf_uir%2Fo",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 647435 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647471 T2745 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 647472 T2745 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 647473 T2745 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49913
[junit4] 2> 647474 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 647475 T2745 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 647475 T2745 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834
[junit4] 2> 647475 T2745 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/'
[junit4] 2> 647484 T2745 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/solr.xml
[junit4] 2> 647494 T2745 oasc.CoreContainer.<init> New CoreContainer 1016064984
[junit4] 2> 647494 T2745 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/]
[junit4] 2> 647495 T2745 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 647495 T2745 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 647495 T2745 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 647495 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 647498 T2745 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 647498 T2745 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 647499 T2745 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 647499 T2745 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37619/solr
[junit4] 2> 647499 T2745 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 647500 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 647501 T2805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11091dd0 name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 647501 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 647502 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 647503 T2807 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a7585f9 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 647503 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 647506 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 647537 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647537 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647537 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647537 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 648508 T2745 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49913_pf_uir%2Fo
[junit4] 2> 648509 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49913_pf_uir%2Fo
[junit4] 2> 648510 T2776 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 648511 T2767 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 648510 T2789 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 648511 T2807 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 648513 T2808 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 648514 T2808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 648514 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 648515 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 648514 T2808 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 648515 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 648516 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 648516 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49913/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49913_pf_uir%2Fo",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 648516 T2768 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 648516 T2768 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 648518 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 648696 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 648696 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 648696 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 648696 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649515 T2808 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 649515 T2808 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1
[junit4] 2> 649515 T2808 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 649516 T2808 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 649516 T2808 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 649517 T2808 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 649517 T2808 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/'
[junit4] 2> 649518 T2808 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/lib/.svn/' to classloader
[junit4] 2> 649518 T2808 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/lib/classes/' to classloader
[junit4] 2> 649518 T2808 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/lib/README' to classloader
[junit4] 2> 649532 T2808 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 649540 T2808 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 649541 T2808 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 649545 T2808 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 649632 T2808 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 649636 T2808 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 649637 T2808 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 649644 T2808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 649646 T2808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 649648 T2808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 649648 T2808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 649648 T2808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 649649 T2808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 649649 T2808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 649649 T2808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 649650 T2808 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 649650 T2808 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2/
[junit4] 2> 649650 T2808 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4a7385
[junit4] 2> 649650 T2808 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2
[junit4] 2> 649651 T2808 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2/index/
[junit4] 2> 649651 T2808 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 649651 T2808 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2/index
[junit4] 2> 649651 T2808 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=2.447265625, floorSegmentMB=1.44921875, forceMergeDeletesPctAllowed=3.3196981008770865, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1240017361539163
[junit4] 2> 649652 T2808 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@22aa45db lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3cebfafb),segFN=segments_1,generation=1}
[junit4] 2> 649652 T2808 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 649655 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 649655 T2808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 649655 T2808 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 649655 T2808 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 649655 T2808 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 649656 T2808 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 649656 T2808 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 649656 T2808 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 649656 T2808 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 649657 T2808 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 649657 T2808 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 649658 T2808 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 649659 T2808 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 649659 T2808 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 649660 T2808 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2564475347007128]
[junit4] 2> 649660 T2808 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@22aa45db lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3cebfafb),segFN=segments_1,generation=1}
[junit4] 2> 649660 T2808 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 649660 T2808 oass.SolrIndexSearcher.<init> Opening Searcher@21096384[collection1] main
[junit4] 2> 649662 T2809 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@21096384[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 649663 T2808 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 649664 T2745 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 649664 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 649664 T2812 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49913/pf_uir/o collection:collection1 shard:shard1
[junit4] 2> 649665 T2812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 649669 T2812 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 649670 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649670 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649670 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649671 T2812 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 649671 T2812 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 649671 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C1055 name=collection1 org.apache.solr.core.SolrCore@ed5308d url=http://127.0.0.1:49913/pf_uir/o/collection1 node=127.0.0.1:49913_pf_uir%2Fo C1055_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:49913/pf_uir/o, core=collection1, node_name=127.0.0.1:49913_pf_uir%2Fo}
[junit4] 2> 649671 T2812 C1055 P49913 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49913/pf_uir/o/collection1/
[junit4] 2> 649672 T2812 C1055 P49913 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 649672 T2812 C1055 P49913 oasc.SyncStrategy.syncToMe http://127.0.0.1:49913/pf_uir/o/collection1/ has no replicas
[junit4] 2> 649672 T2812 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49913/pf_uir/o/collection1/ shard1
[junit4] 2> 649672 T2812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 649673 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649675 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649678 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649782 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649782 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649782 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649782 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649826 T2812 oasc.ZkController.register We are http://127.0.0.1:49913/pf_uir/o/collection1/ and leader is http://127.0.0.1:49913/pf_uir/o/collection1/
[junit4] 2> 649826 T2812 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49913/pf_uir/o
[junit4] 2> 649826 T2812 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 649827 T2812 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 649827 T2812 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 649827 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649828 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649828 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649828 T2812 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 649829 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 649829 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:49913/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49913_pf_uir%2Fo",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 649830 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649867 T2745 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 649868 T2745 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 649870 T2745 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60527
[junit4] 2> 649870 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 649871 T2745 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 649871 T2745 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229
[junit4] 2> 649871 T2745 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/'
[junit4] 2> 649887 T2745 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/solr.xml
[junit4] 2> 649892 T2745 oasc.CoreContainer.<init> New CoreContainer 1546132380
[junit4] 2> 649893 T2745 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/]
[junit4] 2> 649893 T2745 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 649893 T2745 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 649893 T2745 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 649894 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 649894 T2745 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 649894 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 649894 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 649895 T2745 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 649895 T2745 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 649896 T2745 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 649896 T2745 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 649897 T2745 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 649897 T2745 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37619/solr
[junit4] 2> 649897 T2745 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 649898 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 649899 T2823 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@715c94cf name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 649899 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 649901 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 649901 T2825 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33846361 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 649902 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 649905 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 649932 T2825 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649932 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649932 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649932 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649932 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 650907 T2745 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60527_pf_uir%2Fo
[junit4] 2> 650908 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60527_pf_uir%2Fo
[junit4] 2> 650910 T2789 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650910 T2767 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650910 T2776 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650910 T2825 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650910 T2807 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650914 T2826 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 650914 T2826 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 650967 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 650967 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 650967 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 650967 T2826 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 650968 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 650969 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:60527/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60527_pf_uir%2Fo",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 650969 T2768 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 650969 T2768 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 650972 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 651074 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651074 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651074 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651074 T2825 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651074 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651968 T2826 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 651968 T2826 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/collection1
[junit4] 2> 651968 T2826 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 651969 T2826 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 651969 T2826 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 651970 T2826 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 651970 T2826 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/collection1/'
[junit4] 2> 651970 T2826 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/collection1/lib/.svn/' to classloader
[junit4] 2> 651970 T2826 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jett
[...truncated too long message...]
111 T2844 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 667111 T2825 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 667111 T2844 C1081 P55406 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@10648e85 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@185599ef),segFN=segments_7,generation=7}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@10648e85 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@185599ef),segFN=segments_8,generation=8}
[junit4] 2> 667112 T2844 C1081 P55406 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 8
[junit4] 2> 667112 T2825 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 667113 T2844 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4/index;done=false>>]
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4/index
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4;done=false>>]
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4
[junit4] 2> 667114 T2844 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 667114 T2844 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 667202 T2866 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4109d128 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 667203 T2866 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=4CDAA86DE1E3E202 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de_CH -Dtests.timezone=Pacific/Auckland -Dtests.file.encoding=UTF-8
[junit4] FAILURE 23.9s J0 | BasicDistributedZkTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: commitWithin did not work expected:<68> but was:<67>
[junit4] > at __randomizedtesting.SeedInfo.seed([4CDAA86DE1E3E202:CD3C267596BC823E]:0)
[junit4] > at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:354)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 667211 T2745 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 23943 T2744 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=de_CH, timezone=Pacific/Auckland
[junit4] 2> NOTE: Linux 3.8.0-36-generic amd64/Oracle Corporation 1.8.0 (64-bit)/cpus=8,threads=1,free=116480720,total=244842496
[junit4] 2> NOTE: All tests run in this JVM: [TestLMJelinekMercerSimilarityFactory, TestRemoteStreaming, TestCoreContainer, SchemaVersionSpecificBehaviorTest, TestFieldTypeCollectionResource, TestHighFrequencyDictionaryFactory, HdfsCollectionsAPIDistributedZkTest, SuggesterFSTTest, TestMergePolicyConfig, UUIDFieldTest, TestWriterPerf, AssignTest, TestDocBasedVersionConstraints, RecoveryZkTest, ZkCLITest, DateMathParserTest, ExternalFileFieldSortTest, CacheHeaderTest, PeerSyncTest, IndexSchemaTest, SynonymTokenizerTest, SuggestComponentTest, TestSolrXMLSerializer, TestManagedSchema, TestSort, UpdateRequestProcessorFactoryTest, TestFreeTextSuggestions, SolrTestCaseJ4Test, TestInfoStreamLogging, ScriptEngineTest, TestSerializedLuceneMatchVersion, TestDefaultSearchFieldResource, SolrInfoMBeanTest, TestRecoveryHdfs, DOMUtilTest, SuggesterTest, CopyFieldTest, SliceStateUpdateTest, TestLFUCache, TestTrie, TestFieldResource, TestShardHandlerFactory, TestDocSet, SearchHandlerTest, OverseerRolesTest, AnalysisAfterCoreReloadTest, EchoParamsTest, TestRandomDVFaceting, SampleTest, TestSchemaSimilarityResource, TestConfig, QueryElevationComponentTest, TestCollationFieldDocValues, ShardSplitTest, SimpleFacetsTest, FileBasedSpellCheckerTest, TestBadConfig, ClusterStateTest, TestSolrDeletionPolicy1, ResourceLoaderTest, SpatialFilterTest, TriLevelCompositeIdRoutingTest, AbstractAnalyticsStatsTest, TestPseudoReturnFields, DistributedQueryComponentCustomSortTest, TestJmxMonitoredMap, TestCollapseQParserPlugin, CoreAdminCreateDiscoverTest, TestJoin, BasicFunctionalityTest, HdfsChaosMonkeySafeLeaderTest, TestFunctionQuery, DeleteShardTest, TestAnalyzeInfixSuggestions, UnloadDistributedZkTest, SpellPossibilityIteratorTest, StatelessScriptUpdateProcessorFactoryTest, LeaderElectionTest, TestImplicitCoreProperties, SOLR749Test, TestDocumentBuilder, CSVRequestHandlerTest, DistributedDebugComponentTest, DistributedTermsComponentTest, LegacyHTMLStripCharFilterTest, TestSchemaResource, TestPostingsSolrHighlighter, TestSolrQueryParserDefaultOperatorResource, TermVectorComponentTest, BasicDistributedZkTest]
[junit4] Completed on J0 in 24.97s, 1 test, 1 failure <<< FAILURES!
[...truncated 651 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:465: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:445: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1274: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 373 suites, 1606 tests, 1 failure, 35 ignored (7 assumptions)
Total time: 47 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-fcs-b129 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-fcs-b129) - Build
# 9596 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9596/
Java: 32bit/jdk1.8.0-fcs-b129 -server -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED: org.apache.solr.client.solrj.SolrExampleBinaryTest.testUnicode
Error Message:
IOException occured when talking to server at: https://127.0.0.1:39662/solr/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:39662/solr/collection1
at __randomizedtesting.SeedInfo.seed([CA32FB715F20BCA7:D3FE4F7BFD9A6D50]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:510)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54)
at org.apache.solr.client.solrj.SolrExampleTests.testUnicode(SolrExampleTests.java:370)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:395)
... 45 more
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testShutdown
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
at __randomizedtesting.SeedInfo.seed([CA32FB715F20BCA7:2944F2E4B85A29D5]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:200)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:222)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
... 47 more
Build Log:
[...truncated 11458 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 5100 T23 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 5100 T23 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 5103 T23 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 5111 T23 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1393373232300
[junit4] 2> 5115 T23 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 5118 T24 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 5217 T23 oasc.ZkTestServer.run start zk server on port:45644
[junit4] 2> 5265 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 71927 T27 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 66639ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 71929 T23 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 71950 T25 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446b843ceb0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 71953 T23 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:45644 45644
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testShutdown -Dtests.seed=CA32FB715F20BCA7 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=Asia/Sakhalin -Dtests.file.encoding=US-ASCII
[junit4] ERROR 67.1s J0 | CloudSolrServerTest.testShutdown <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([CA32FB715F20BCA7:2944F2E4B85A29D5]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:200)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:77)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:222)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
[junit4] > ... 47 more
[junit4] 2> 72239 T23 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1393373299430
[junit4] 2> 72242 T23 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 72243 T31 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 72343 T23 oasc.ZkTestServer.run start zk server on port:35609
[junit4] 2> 72345 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72389 T37 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1af289f name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72389 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72391 T23 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 72412 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72413 T39 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2305a name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72414 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72418 T23 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 72424 T23 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 72429 T23 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 72432 T23 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 72438 T23 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 72445 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 72452 T23 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 72452 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 72456 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 72457 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 72457 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 72457 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 72458 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 72458 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 72458 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 72459 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 72459 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 72463 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72467 T41 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cd7e13 name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72468 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72690 T23 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 73021 T23 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 73079 T23 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:33655
[junit4] 2> 73141 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 73142 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 73142 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659
[junit4] 2> 73143 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/'
[junit4] 2> 73163 T23 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/solr.xml
[junit4] 2> 73193 T23 oasc.CoreContainer.<init> New CoreContainer 26862557
[junit4] 2> 73194 T23 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/]
[junit4] 2> 73195 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 73196 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 73196 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 73197 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 73198 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 73200 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 73201 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 73202 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 73202 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 73226 T23 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 73226 T23 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 73227 T23 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 73228 T23 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35609/solr
[junit4] 2> 73248 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 73258 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 73269 T53 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f8c512 name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 73274 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 73294 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 73308 T55 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@503b85 name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 73308 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 73315 T23 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 73323 T23 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 73345 T23 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 73363 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33655_
[junit4] 2> 73366 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33655_
[junit4] 2> 73371 T23 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 73375 T23 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 73386 T23 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:33655_
[junit4] 2> 73387 T23 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 73390 T23 oasc.Overseer.start Overseer (id=91316112551903236-127.0.0.1:33655_-n_0000000000) starting
[junit4] 2> 73450 T23 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 73494 T57 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 73495 T23 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 73495 T56 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 73502 T57 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 73508 T23 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 73510 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 73528 T58 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 73529 T58 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 73540 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 73540 T58 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 73544 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 73546 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33655",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33655_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 73547 T56 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 73558 T56 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 73565 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 73571 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 74541 T58 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 74541 T58 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/collection1
[junit4] 2> 74541 T58 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 74542 T58 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 74543 T58 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 74544 T58 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 74545 T58 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/collection1/'
[junit4] 2> 74580 T58 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 74590 T58 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 74597 T58 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 74615 T58 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 74653 T58 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 74659 T58 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableLongField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 74667 T58 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableFloatField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 74671 T58 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableDoubleField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 74895 T58 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 74900 T58 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 74904 T58 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 74904 T58 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 74909 T58 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 74910 T58 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/
[junit4] 2> 74910 T58 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 74911 T58 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data
[junit4] 2> 74912 T58 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index/
[junit4] 2> 74912 T58 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 74913 T58 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index
[junit4] 2> 74915 T58 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1a928d1 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 74915 T58 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74917 T58 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 74917 T58 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 74918 T58 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 74918 T58 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 74924 T58 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 74925 T58 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 74925 T58 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 74925 T58 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 74928 T58 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 74931 T58 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 74936 T58 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 74937 T58 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1a928d1 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 74938 T58 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74938 T58 oass.SolrIndexSearcher.<init> Opening Searcher@17e7fc3[collection1] main
[junit4] 2> 74946 T59 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17e7fc3[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 74947 T58 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 74953 T62 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:33655 collection:control_collection shard:shard1
[junit4] 2> 74953 T23 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 74954 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 74964 T62 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 74983 T62 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 74989 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74990 T62 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 74990 T62 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@498617 url=https://127.0.0.1:33655/collection1 node=127.0.0.1:33655_ C2_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:33655, core=collection1, node_name=127.0.0.1:33655_}
[junit4] 2> 74992 T62 C2 P33655 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:33655/collection1/
[junit4] 2> 74993 T62 C2 P33655 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 74993 T62 C2 P33655 oasc.SyncStrategy.syncToMe https://127.0.0.1:33655/collection1/ has no replicas
[junit4] 2> 74993 T62 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:33655/collection1/ shard1
[junit4] 2> 74994 T62 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 74997 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 75015 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 75018 T64 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ecdae name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 75020 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 75022 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 75026 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75033 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75035 T23 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 75068 T23 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 75072 T23 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 75077 T23 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 75081 T23 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:59298
[junit4] 2> 75083 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 75084 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 75084 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224
[junit4] 2> 75085 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/'
[junit4] 2> 75099 T23 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/solr.xml
[junit4] 2> 75116 T23 oasc.CoreContainer.<init> New CoreContainer 17678257
[junit4] 2> 75116 T23 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/]
[junit4] 2> 75117 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 75117 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 75117 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 75118 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 75118 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 75118 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 75119 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 75119 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 75119 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 75123 T23 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 75124 T23 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 75124 T23 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 75125 T23 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35609/solr
[junit4] 2> 75125 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 75126 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 75152 T75 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55f571 name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 75153 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 75154 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75155 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 75155 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 75162 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 75167 T77 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1eb0b7b name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 75167 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 75181 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 75186 T62 oasc.ZkController.register We are https://127.0.0.1:33655/collection1/ and leader is https://127.0.0.1:33655/collection1/
[junit4] 2> 75187 T62 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:33655
[junit4] 2> 75187 T62 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 75187 T62 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 75187 T62 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 75189 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75190 T62 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 75192 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 75193 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:33655",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33655_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 75197 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75300 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 75300 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 75300 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 76186 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59298_
[junit4] 2> 76187 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59298_
[junit4] 2> 76189 T77 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 76189 T64 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 76190 T55 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 76199 T78 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 76200 T78 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 76202 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76202 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76202 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76202 T78 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 76203 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 76204 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:59298",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:59298_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 76205 T56 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 76205 T56 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 76208 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76314 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 76314 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 76314 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77206 T78 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 77206 T78 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/collection1
[junit4] 2> 77206 T78 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 77207 T78 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 77207 T78 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 77209 T78 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 77209 T78 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/collection1/'
[junit4] 2> 77240 T78 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 77268 T78 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 77272 T78 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 77290 T78 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 77503 T78 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 77505 T78 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 77507 T78 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 77507 T78 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 77508 T78 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 77508 T78 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/
[junit4] 2> 77509 T78 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 77510 T78 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1
[junit4] 2> 77510 T78 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index/
[junit4] 2> 77510 T78 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 77510 T78 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index
[junit4] 2> 77512 T78 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4a20a1 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 77512 T78 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 77513 T78 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 77513 T78 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 77513 T78 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 77513 T78 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 77513 T78 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 77516 T78 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 77516 T78 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 77516 T78 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 77518 T78 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 77520 T78 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 77520 T78 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 77520 T78 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4a20a1 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 77521 T78 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 77521 T78 oass.SolrIndexSearcher.<init> Opening Searcher@1780fa7[collection1] main
[junit4] 2> 77522 T79 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1780fa7[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 77522 T78 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 77523 T23 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 77523 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 77523 T82 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:59298 collection:collection1 shard:shard2
[junit4] 2> 77525 T82 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 77531 T82 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 77533 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77533 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77533 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77534 T82 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 77534 T82 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@16dc16 url=https://127.0.0.1:59298/collection1 node=127.0.0.1:59298_ C3_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:59298, core=collection1, node_name=127.0.0.1:59298_}
[junit4] 2> 77535 T82 C3 P59298 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:59298/collection1/
[junit4] 2> 77535 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77536 T82 C3 P59298 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 77536 T82 C3 P59298 oasc.SyncStrategy.syncToMe https://127.0.0.1:59298/collection1/ has no replicas
[junit4] 2> 77536 T82 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:59298/collection1/ shard2
[junit4] 2> 77536 T82 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 77541 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77548 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77551 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77560 T23 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 77561 T23 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 77565 T23 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 77570 T23 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:54886
[junit4] 2> 77572 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 77572 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 77573 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713
[junit4] 2> 77573 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/'
[junit4] 2> 77584 T23 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/solr.xml
[junit4] 2> 77599 T23 oasc.CoreContainer.<init> New CoreContainer 12936508
[junit4] 2> 77603 T23 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/]
[junit4] 2> 77604 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 77604 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 77605 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 77605 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 77606 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 77606 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 77607 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 77607 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 77607 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 77618 T23 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 77618 T23 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 77619 T23 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 77619 T23 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35609/solr
[junit4] 2> 77620 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 77621 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77623 T93 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12fa9c0 name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77624 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77626 T32 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446b8542ed0008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 77627 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77640 T95 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@169237c name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77640 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77649 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 77655 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77655 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77655 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77655 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77699 T82 oasc.ZkController.register We are https://127.0.0.1:59298/collection1/ and leader is https://127.0.0.1:59298/collection1/
[junit4] 2> 77699 T82 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:59298
[junit4] 2> 77700 T82 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 77700 T82 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 77700 T82 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 77701 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77701 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77701 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77702 T82 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77703 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77704 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:59298",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:59298_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 77707 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77809 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77809 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77810 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77810 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 78653 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54886_
[junit4] 2> 78655 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54886_
[junit4] 2> 78657 T64 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 78657 T95 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 78657 T55 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 78657 T77 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 78664 T96 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 78665 T96 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 78666 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78666 T96 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 78666 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78666 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78668 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 78668 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:54886",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54886_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 78669 T56 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 78669 T56 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 78671 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78774 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78774 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78774 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78774 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 79666 T96 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 79667 T96 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/collection1
[junit4] 2> 79667 T96 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 79668 T96 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 79668 T96 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 79669 T96 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 79669 T96 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/collection1/'
[junit4] 2> 79688 T96 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 79699 T96 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 79700 T96 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 79713 T96 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 79845 T96 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 79847 T96 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 79850 T96 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 79850 T96 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 79851 T96 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 79852 T96 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/
[junit4] 2> 79852 T96 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 79853 T96 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2
[junit4] 2> 79854 T96 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index/
[junit4] 2> 79854 T96 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 79854 T96 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index
[junit4] 2> 79857 T96 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1ca5b09 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 79857 T96 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79860 T96 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 79861 T96 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 79862 T96 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 79862 T96 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 79862 T96 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 79863 T96 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 79863 T96 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 79863 T96 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 79866 T96 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 79867 T96 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 79868 T96 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 79869 T96 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1ca5b09 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 79869 T96 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79870 T96 oass.SolrIndexSearcher.<init> Opening Searcher@1f739c4[collection1] main
[junit4] 2> 79871 T96 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 79872 T97 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f739c4[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 79872 T23 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 79872 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 79874 T100 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:54886 collection:collection1 shard:shard1
[junit4] 2> 79876 T100 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 79891 T100 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 79894 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79894 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79894 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79894 T100 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 79895 T100 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@125d3d8 url=https://127.0.0.1:54886/collection1 node=127.0.0.1:54886_ C4_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:54886, core=collection1, node_name=127.0.0.1:54886_}
[junit4] 2> 79895 T100 C4 P54886 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:54886/collection1/
[junit4] 2> 79895 T100 C4 P54886 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 79895 T100 C4 P54886 oasc.SyncStrategy.syncToMe https://127.0.0.1:54886/collection1/ has no replicas
[junit4] 2> 79896 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 79896 T100 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:54886/collection1/ shard1
[junit4] 2> 79896 T100 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 79900 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79903 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79907 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79917 T23 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 79918 T23 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 79927 T23 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 79932 T23 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:56893
[junit4] 2> 79935 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 79935 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 79936 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063
[junit4] 2> 79936 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/'
[junit4] 2> 79959 T23 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/solr.xml
[junit4] 2> 79981 T23 oasc.CoreContainer.<init> New CoreContainer 9700943
[junit4] 2> 79982 T23 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/]
[junit4] 2> 79982 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 79983 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 79983 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 79984 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 79984 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 79985 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 79985 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 79986 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 79986 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 79993 T23 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 79993 T23 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 79994 T23 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 79994 T23 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35609/solr
[junit4] 2> 79995 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 80004 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 80007 T111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35bb65 name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 80008 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 80012 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80012 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80013 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80013 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80015 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 80020 T113 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@189522c name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 80020 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 80026 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 80054 T100 oasc.ZkController.register We are https://127.0.0.1:54886/collection1/ and leader is https://127.0.0.1:54886/collection1/
[junit4] 2> 80055 T100 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:54886
[junit4] 2> 80055 T100 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 80055 T100 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 80055 T100 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 80057 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80058 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80058 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80058 T100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 80060 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 80061 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:54886",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54886_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 80064 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80167 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80167 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80167 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80167 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80167 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 81030 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56893_
[junit4] 2> 81031 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56893_
[junit4] 2> 81034 T95 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81034 T55 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81035 T77 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81034 T64 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81034 T113 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81045 T114 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 81045 T114 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 81046 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81046 T114 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 81047 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81047 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81048 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 81049 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:56893",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:56893_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 81049 T56 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 81050 T56 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 81052 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81155 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 81155 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 81155 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 81155 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 81155 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 82047 T114 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 82047 T114 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/collection1
[junit4] 2> 82047 T114 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 82048 T114 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 82048 T114 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 82049 T114 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 82050 T114 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/collection1/'
[junit4] 2> 82074 T114 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 82088 T114 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 82089 T114 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 82100 T114 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 82283 T114 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 82284 T114 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 82286 T114 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 82286 T114 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 82286 T114 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 82287 T114 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty3/
[junit4] 2> 82287 T114 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 82288 T114 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty3
[junit4] 2> 82288 T114 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty3/index/
[junit4] 2> 82288 T114 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 82288 T114 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429
[...truncated too long message...]
te params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 13
[junit4] 2> 303513 T82 C34 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=1
[junit4] 2> 303515 T81 C34 oasc.SolrCore.execute [collection1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 303516 T81 C34 oasc.SolrCore.execute [collection1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 303516 T74 oas.SolrTestCaseJ4.tearDown ###Ending testPingHandler
[junit4] 2> 303539 T74 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=28609177
[junit4] 2> 303540 T74 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@16b4852
[junit4] 2> 303540 T74 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=62,autocommit maxTime=15000ms,autocommits=1,soft autocommits=3,optimizes=2,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=1,cumulative_adds=248,cumulative_deletesById=5,cumulative_deletesByQuery=30,cumulative_errors=1,transaction_logs_total_size=1243,transaction_logs_total_number=10}
[junit4] 2> 303541 T74 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 303541 T74 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 303541 T74 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 303542 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: now flush at close waitForMerges=true
[junit4] 2> 303542 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: start flush: applyAllDeletes=true
[junit4] 2> 303542 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: index before flush
[junit4] 2> 303542 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker startFullFlush
[junit4] 2> 303543 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 303543 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker finishFullFlush success=true
[junit4] 2> 303543 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: apply all deletes during flush
[junit4] 2> 303544 T74 C34 oasu.LoggingInfoStream.message [BD][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: prune sis=org.apache.lucene.index.SegmentInfos@bedfe minGen=9223372036854775807 packetCount=0
[junit4] 2> 303544 T74 C34 oasu.LoggingInfoStream.message [CMS][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: now merge
[junit4] 2> 303545 T74 C34 oasu.LoggingInfoStream.message [CMS][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: index:
[junit4] 2> 303545 T74 C34 oasu.LoggingInfoStream.message [CMS][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: no more merges pending; now return
[junit4] 2> 303545 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: waitForMerges
[junit4] 2> 303545 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: waitForMerges done
[junit4] 2> 303546 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: now call final commit()
[junit4] 2> 303546 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: start
[junit4] 2> 303546 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: enter lock
[junit4] 2> 303547 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: now prepare
[junit4] 2> 303547 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: prepareCommit: flush
[junit4] 2> 303548 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: index before flush
[junit4] 2> 303548 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker startFullFlush
[junit4] 2> 303548 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 303549 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: apply all deletes during flush
[junit4] 2> 303549 T74 C34 oasu.LoggingInfoStream.message [BD][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: prune sis=org.apache.lucene.index.SegmentInfos@bedfe minGen=9223372036854775807 packetCount=0
[junit4] 2> 303549 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker finishFullFlush success=true
[junit4] 2> 303549 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: startCommit(): start
[junit4] 2> 303550 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: skip startCommit(): no changes pending
[junit4] 2> 303550 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: pendingCommit == null; skip
[junit4] 2> 303550 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: done
[junit4] 2> 303550 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: at close:
[junit4] 2> 303551 T74 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 303552 T74 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 303552 T74 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760;done=false>>]
[junit4] 2> 303552 T74 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760
[junit4] 2> 303552 T74 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760/index;done=false>>]
[junit4] 2> 303553 T74 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760/index
[junit4] 2> 303553 T74 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4] 2> 303605 T74 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=sr_ME, timezone=US/East-Indiana
[junit4] 2> NOTE: Linux 3.8.0-36-generic i386/Oracle Corporation 1.8.0 (32-bit)/cpus=8,threads=1,free=32826296,total=64880640
[junit4] 2> NOTE: All tests run in this JVM: [TestNamedListCodec, SolrQueryTest, MergeIndexesEmbeddedTest, SolrExampleStreamingTest, SolrDocumentTest, SolrExampleBinaryTest]
[junit4] Completed on J1 in 208.08s, 26 tests, 1 error <<< FAILURES!
[...truncated 23 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:465: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:445: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1274: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 48 suites, 280 tests, 2 errors
Total time: 56 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-fcs-b129 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60-ea-b04) -
Build # 9595 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9595/
Java: 32bit/jdk1.7.0_60-ea-b04 -server -XX:+UseSerialGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([7251B54D221CEEDC]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([7251B54D221CEEDC]:0)
Build Log:
[...truncated 11154 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> 529169 T2151 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 529169 T2151 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /khv/wa
[junit4] 2> 529173 T2151 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeySafeLeaderTest-1393357415240
[junit4] 2> 529173 T2151 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 529174 T2152 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 529274 T2151 oasc.ZkTestServer.run start zk server on port:58172
[junit4] 2> 529275 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529342 T2158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1229554 name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529343 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529343 T2151 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 529351 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529353 T2160 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c20f96 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529353 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529354 T2151 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 529358 T2151 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 529362 T2151 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 529366 T2151 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 529370 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 529371 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 529378 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 529379 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 529385 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 529385 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 529390 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 529391 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 529396 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 529397 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 529402 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 529403 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 529408 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 529408 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 529413 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 529414 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 529419 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 529419 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 529423 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 529424 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 529428 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 529429 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 529435 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529437 T2162 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2955eb name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529437 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529646 T2151 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 529649 T2151 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 529665 T2151 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:45401
[junit4] 2> 529666 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 529667 T2151 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 529667 T2151 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508
[junit4] 2> 529667 T2151 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/'
[junit4] 2> 529676 T2151 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/solr.xml
[junit4] 2> 529697 T2151 oasc.CoreContainer.<init> New CoreContainer 28838409
[junit4] 2> 529697 T2151 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/]
[junit4] 2> 529698 T2151 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 529698 T2151 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 529698 T2151 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 529699 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 529699 T2151 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 529699 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 529700 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 529700 T2151 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 529700 T2151 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 529703 T2151 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 529704 T2151 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 529704 T2151 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 529704 T2151 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58172/solr
[junit4] 2> 529705 T2151 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 529705 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529707 T2173 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fd7f6 name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529707 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529710 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529711 T2175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1243546 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529711 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529713 T2151 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 529715 T2151 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 529719 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 529720 T2151 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45401_khv%2Fwa
[junit4] 2> 529722 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45401_khv%2Fwa
[junit4] 2> 529726 T2151 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 529727 T2151 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 529730 T2151 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:45401_khv%2Fwa
[junit4] 2> 529731 T2151 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 529732 T2151 oasc.Overseer.start Overseer (id=91315071565365252-127.0.0.1:45401_khv%2Fwa-n_0000000000) starting
[junit4] 2> 529735 T2151 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 529738 T2177 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 529738 T2151 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 529739 T2177 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 529743 T2151 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 529745 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 529747 T2176 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 529749 T2178 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 529750 T2178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 529750 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 529750 T2178 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 529752 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 529752 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45401/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:45401_khv%2Fwa",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 529752 T2176 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 529753 T2176 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 529754 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 529755 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 530751 T2178 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 530751 T2178 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1
[junit4] 2> 530752 T2178 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 530753 T2178 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 530753 T2178 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 530755 T2178 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 530756 T2178 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/'
[junit4] 2> 530756 T2178 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/lib/.svn/' to classloader
[junit4] 2> 530757 T2178 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/lib/classes/' to classloader
[junit4] 2> 530757 T2178 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/lib/README' to classloader
[junit4] 2> 530803 T2178 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 530824 T2178 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 530826 T2178 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 530831 T2178 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 531012 T2178 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 531013 T2178 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 531014 T2178 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 531019 T2178 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 531059 T2178 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 531060 T2178 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/
[junit4] 2> 531060 T2178 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f250ac
[junit4] 2> 531061 T2178 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data
[junit4] 2> 531062 T2178 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index/
[junit4] 2> 531062 T2178 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 531063 T2178 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index
[junit4] 2> 531064 T2178 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=97.7001953125, floorSegmentMB=0.65234375, forceMergeDeletesPctAllowed=25.842840998478867, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 531068 T2178 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 531068 T2178 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 531072 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 531072 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 531073 T2178 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 531073 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 531074 T2178 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 531074 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 531075 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 531075 T2178 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 531075 T2178 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 531076 T2178 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 531077 T2178 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 531077 T2178 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 531077 T2178 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 531078 T2178 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 531078 T2178 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 531079 T2178 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 531080 T2178 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 531083 T2178 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 531086 T2178 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 531087 T2178 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 531088 T2178 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 531089 T2178 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 531090 T2178 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 531090 T2178 oass.SolrIndexSearcher.<init> Opening Searcher@2ff84c[collection1] main
[junit4] 2> 531092 T2179 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2ff84c[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 531094 T2178 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 531094 T2151 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 531095 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 531094 T2182 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:45401/khv/wa collection:control_collection shard:shard1
[junit4] 2> 531097 T2182 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 531100 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 531101 T2184 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a32201 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 531102 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 531104 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 531108 T2182 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 531109 T2151 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 531111 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531111 T2182 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 531112 T2182 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2190 name=collection1 org.apache.solr.core.SolrCore@14a1ca0 url=https://127.0.0.1:45401/khv/wa/collection1 node=127.0.0.1:45401_khv%2Fwa C2190_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:45401/khv/wa, core=collection1, node_name=127.0.0.1:45401_khv%2Fwa}
[junit4] 2> 531112 T2182 C2190 P45401 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:45401/khv/wa/collection1/
[junit4] 2> 531113 T2182 C2190 P45401 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 531113 T2182 C2190 P45401 oasc.SyncStrategy.syncToMe https://127.0.0.1:45401/khv/wa/collection1/ has no replicas
[junit4] 2> 531113 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 531113 T2182 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:45401/khv/wa/collection1/ shard1
[junit4] 2> 531114 T2182 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 531116 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531118 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531124 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531228 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 531228 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 531270 T2182 oasc.ZkController.register We are https://127.0.0.1:45401/khv/wa/collection1/ and leader is https://127.0.0.1:45401/khv/wa/collection1/
[junit4] 2> 531270 T2182 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:45401/khv/wa
[junit4] 2> 531270 T2182 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 531270 T2182 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 531270 T2182 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 531271 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531271 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531271 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531272 T2182 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 531273 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 531273 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:45401/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:45401_khv%2Fwa",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 531275 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531314 T2151 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 531315 T2151 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 531318 T2151 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 531319 T2151 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:34509
[junit4] 2> 531322 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 531322 T2151 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 531323 T2151 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177
[junit4] 2> 531324 T2151 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/'
[junit4] 2> 531342 T2151 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/solr.xml
[junit4] 2> 531364 T2151 oasc.CoreContainer.<init> New CoreContainer 1726962
[junit4] 2> 531365 T2151 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/]
[junit4] 2> 531365 T2151 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 531365 T2151 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 531366 T2151 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 531366 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 531366 T2151 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 531367 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 531367 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 531367 T2151 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 531367 T2151 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 531371 T2151 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 531371 T2151 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 531371 T2151 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 531372 T2151 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58172/solr
[junit4] 2> 531372 T2151 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 531373 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 531374 T2195 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b12097 name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 531375 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 531378 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 531378 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 531379 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 531380 T2197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e0dcf6 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 531380 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 531388 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 532393 T2151 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34509_khv%2Fwa
[junit4] 2> 532394 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34509_khv%2Fwa
[junit4] 2> 532397 T2184 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 532397 T2175 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 532397 T2197 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 532402 T2198 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 532403 T2198 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 532404 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 532404 T2198 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 532404 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 532404 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 532405 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 532406 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34509/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34509_khv%2Fwa",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 532406 T2176 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 532406 T2176 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 532408 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 532513 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 532513 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 532513 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 533404 T2198 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 533404 T2198 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1
[junit4] 2> 533405 T2198 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 533406 T2198 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 533406 T2198 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 533408 T2198 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 533408 T2198 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/'
[junit4] 2> 533409 T2198 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/lib/.svn/' to classloader
[junit4] 2> 533410 T2198 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/lib/classes/' to classloader
[junit4] 2> 533410 T2198 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/lib/README' to classloader
[junit4] 2> 533477 T2198 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 533501 T2198 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 533503 T2198 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 533508 T2198 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 533699 T2198 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 533700 T2198 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 533701 T2198 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 533705 T2198 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 533724 T2198 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 533724 T2198 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/
[junit4] 2> 533724 T2198 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f250ac
[junit4] 2> 533725 T2198 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1
[junit4] 2> 533725 T2198 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index/
[junit4] 2> 533726 T2198 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 533727 T2198 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index
[junit4] 2> 533728 T2198 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=97.7001953125, floorSegmentMB=0.65234375, forceMergeDeletesPctAllowed=25.842840998478867, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 533730 T2198 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 533730 T2198 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 533733 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 533733 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 533733 T2198 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 533734 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 533734 T2198 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 533734 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 533734 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 533735 T2198 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 533735 T2198 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 533735 T2198 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 533735 T2198 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 533736 T2198 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 533736 T2198 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 533736 T2198 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 533736 T2198 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 533737 T2198 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 533737 T2198 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 533740 T2198 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 533743 T2198 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 533743 T2198 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 533744 T2198 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 533745 T2198 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 533746 T2198 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 533746 T2198 oass.SolrIndexSearcher.<init> Opening Searcher@18216e3[collection1] main
[junit4] 2> 533750 T2199 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18216e3[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 533753 T2198 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 533754 T2151 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 533754 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 533754 T2202 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:34509/khv/wa collection:collection1 shard:shard3
[junit4] 2> 533756 T2202 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 533764 T2202 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
[junit4] 2> 533767 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533767 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533767 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533767 T2202 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 533768 T2202 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2191 name=collection1 org.apache.solr.core.SolrCore@8ac754 url=https://127.0.0.1:34509/khv/wa/collection1 node=127.0.0.1:34509_khv%2Fwa C2191_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:34509/khv/wa, core=collection1, node_name=127.0.0.1:34509_khv%2Fwa}
[junit4] 2> 533768 T2202 C2191 P34509 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:34509/khv/wa/collection1/
[junit4] 2> 533768 T2202 C2191 P34509 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 533768 T2202 C2191 P34509 oasc.SyncStrategy.syncToMe https://127.0.0.1:34509/khv/wa/collection1/ has no replicas
[junit4] 2> 533768 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 533768 T2202 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:34509/khv/wa/collection1/ shard3
[junit4] 2> 533769 T2202 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 533770 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533772 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533774 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533879 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 533879 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 533879 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 533923 T2202 oasc.ZkController.register We are https://127.0.0.1:34509/khv/wa/collection1/ and leader is https://127.0.0.1:34509/khv/wa/collection1/
[junit4] 2> 533923 T2202 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:34509/khv/wa
[junit4] 2> 533923 T2202 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 533923 T2202 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 533924 T2202 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 533924 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533925 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533925 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533925 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 533926 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 533926 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:34509/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34509_khv%2Fwa",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 533928 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533954 T2151 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 533955 T2151 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 533958 T2151 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 533960 T2151 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:35642
[junit4] 2> 533963 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 533964 T2151 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 533964 T2151 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823
[junit4] 2> 533965 T2151 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/'
[junit4] 2> 533974 T2151 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/solr.xml
[junit4] 2> 533994 T2151 oasc.CoreContainer.<init> New CoreContainer 22431363
[junit4] 2> 533995 T2151 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/]
[junit4] 2> 533995 T2151 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 533996 T2151 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 533996 T2151 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 533996 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 533996 T2151 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 533997 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 533997 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 533997 T2151 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 533997 T2151 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 534000 T2151 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 534001 T2151 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 534001 T2151 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 534001 T2151 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58172/solr
[junit4] 2> 534002 T2151 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 534002 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 534004 T2213 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@126178f name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 534005 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 534007 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 534008 T2215 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@189ea73 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 534008 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 534014 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 534031 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 534031 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 534031 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 534031 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 535016 T2151 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35642_khv%2Fwa
[junit4] 2> 535018 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35642_khv%2Fwa
[junit4] 2> 535020 T2175 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 535020 T2184 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 535020 T2197 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 535020 T2215 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 535027 T2216 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 535027 T2216 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 535028 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 535028 T2216 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 535028 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 535029 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 535030 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 535030 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:35642/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35642_khv%2Fwa",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 535030 T2176 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 535031 T2176 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 535034 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 535137 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 535137 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 535137 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 535137 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536029 T2216 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 536029 T2216 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1
[junit4] 2> 536029 T2216 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 536031 T2216 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 536031 T2216 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 536033 T2216 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 536033 T2216 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/'
[junit4] 2> 536034 T2216 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/lib/.svn/' to classloader
[junit4] 2> 536035 T2216 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/lib/classes/' to classloader
[junit4] 2> 536035 T2216 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/lib/README' to classloader
[junit4] 2> 536081 T2216 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 536104 T2216 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 536106 T2216 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 536111 T2216 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 536326 T2216 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 536328 T2216 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 536329 T2216 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 536333 T2216 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 536355 T2216 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 536356 T2216 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/
[junit4] 2> 536356 T2216 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f250ac
[junit4] 2> 536357 T2216 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2
[junit4] 2> 536357 T2216 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index/
[junit4] 2> 536358 T2216 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 536359 T2216 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index
[junit4] 2> 536359 T2216 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=97.7001953125, floorSegmentMB=0.65234375, forceMergeDeletesPctAllowed=25.842840998478867, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 536363 T2216 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 536363 T2216 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 536368 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 536369 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 536369 T2216 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 536369 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 536370 T2216 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 536370 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 536371 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 536371 T2216 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 536372 T2216 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 536373 T2216 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 536373 T2216 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 536374 T2216 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 536374 T2216 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 536375 T2216 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 536375 T2216 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 536376 T2216 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 536377 T2216 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 536381 T2216 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 536384 T2216 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 536384 T2216 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 536385 T2216 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 536387 T2216 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 536387 T2216 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 536387 T2216 oass.SolrIndexSearcher.<init> Opening Searcher@4488d8[collection1] main
[junit4] 2> 536389 T2217 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4488d8[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 536390 T2216 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 536391 T2151 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 536392 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 536392 T2220 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:35642/khv/wa collection:collection1 shard:shard2
[junit4] 2> 536394 T2220 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 536401 T2220 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 536404 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536405 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536405 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536405 T2220 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 536406 T2220 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2192 name=collection1 org.apache.solr.core.SolrCore@894fe3 url=https://127.0.0.1:35642/khv/wa/collection1 node=127.0.0.1:35642_khv%2Fwa C2192_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:35642/khv/wa, core=collection1, node_name=127.0.0.1:35642_khv%2Fwa}
[junit4] 2> 536406 T2220 C2192 P35642 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:35642/khv/wa/collection1/
[junit4] 2> 536406 T2220 C2192 P35642 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 536406 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 536407 T2220 C2192 P35642 oasc.SyncStrategy.syncToMe https://127.0.0.1:35642/khv/wa/collection1/ has no replicas
[junit4] 2> 536407 T2220 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:35642/khv/wa/collection1/ shard2
[junit4] 2> 536407 T2220 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 536409 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536412 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536416 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536519 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536519 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536519 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536519 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536563 T2220 oasc.ZkController.register We are https://127.0.0.1:35642/khv/wa/collection1/ and leader is https://127.0.0.1:35642/khv/wa/collection1/
[junit4] 2> 536563 T2220 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:35642/khv/wa
[junit4] 2> 536563 T2220 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 536563 T2220 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 536563 T2220 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 536564 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536564 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536565 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536565 T2220 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 536566 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 536566 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:35642/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35642_khv%2Fwa",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 536568 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536598 T2151 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 536599 T2151 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 536603 T2151 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 536605 T2151 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55287
[junit4] 2> 536608 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 536609 T2151 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 536609 T2151 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461
[junit4] 2> 536610 T2151 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/'
[junit4] 2> 536628 T2151 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/solr.xml
[junit4] 2> 536648 T2151 oasc.CoreContainer.<init> New CoreContainer 15609678
[junit4] 2> 536648 T2151 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/]
[junit4] 2> 536649 T2151 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 536649 T2151 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 536649 T2151 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 536650 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 536650 T2151 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 536650 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 536650 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 536651 T2151 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 536651 T2151 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 536654 T2151 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 536655 T2151 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 536655 T2151 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 536655 T2151 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58172/solr
[junit4] 2> 536656 T2151 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 536656 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 536659 T2231 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1533afd name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 536659 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 536663 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 536664 T2233 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1edbb54 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 536664 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 536668 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 536670 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536670 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536670 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536670 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536670 T2233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 537676 T2151 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55287_khv%2Fwa
[junit4] 2> 537679 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55287_khv%2Fwa
[junit4] 2> 537683 T2184 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537683 T2215 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537683 T2197 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537684 T2175 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537683 T2233 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537695 T2234 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 537696 T2234 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 537698 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 537698 T2234 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 537698 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 537699 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 537700 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 537701 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:55287/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:55287_khv%2Fwa",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 537701 T2176 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 537702 T2176 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 537706 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 537809 T2233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 537809 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 537809 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 537809 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 537809 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 538698 T2234 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 538699 T2234 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1
[junit4] 2> 538699 T2234 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 538700 T2234 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 538701 T2234 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 538703 T2234 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 538703 T2234 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/'
[junit4] 2> 538704 T2234 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/lib/.svn/' to classloader
[junit4] 2> 538705 T2234 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/lib/classes/' to classloader
[junit4] 2> 538705 T2234 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/lib/README' to classloader
[junit4] 2> 538748 T2234 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 538768 T2234 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 538770 T2234 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 538775 T2234 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 538968 T2234 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 538970 T2234 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 538971 T2234 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 538974 T2234 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 538989 T2234 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 538990 T2234 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/
[junit4] 2> 538990 T2234 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f250ac
[junit4] 2> 538990 T2234 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3
[junit4] 2> 538991 T2234 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/index/
[junit4] 2> 538991 T2234 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 538992 T2234 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/index
[junit4] 2> 538992 T2234 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=97.7001953125, floorSegmentMB=0.65234375, forceMergeDeletesPctAllowed=25.842840998478867, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 538995 T2234 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/index,segFN=segments_1,generation=1}
[junit4] 2> 538995 T2234 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 538999 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 538999 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 538999 T2234 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 539000 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 539000 T2234 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 539000 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 539001 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 539001 T2234 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 539001 T2234 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 539002 T2234 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 539002 T2234 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 539003 T2234 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 539003 T2234 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 53
[...truncated too long message...]
2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 67) Thread[id=3061, name=Thread-1572, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4] 2> at org.apache.solr.core.CloserThread.run(CoreContainer.java:1018)
[junit4] 2> 68) Thread[id=3235, name=qtp15937844-3235 Selector0, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:193)
[junit4] 2> at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
[junit4] 2> at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:78)
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4] 2> at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:536)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 69) Thread[id=3050, name=qtp9333592-3050 Acceptor0 SslSelectChannelConnector@127.0.0.1:35928, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 70) Thread[id=3051, name=qtp9333592-3051 Acceptor1 SslSelectChannelConnector@127.0.0.1:35928, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 71) Thread[id=2595, name=qtp20914976-2595 Selector1, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.nio.ch.IOUtil.drain(Native Method)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4] 2> at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:536)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 72) Thread[id=2597, name=qtp20914976-2597 Acceptor1 SslSelectChannelConnector@127.0.0.1:52388, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 73) Thread[id=2934, name=Thread-1492, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4] 2> at org.apache.solr.core.CloserThread.run(CoreContainer.java:1018)
[junit4] 2> 74) Thread[id=3066, name=HashSessionScavenger-102, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4] 2> NOTE: test params are: codec=Lucene46: {a_si=PostingsFormat(name=NestedPulsing), _version_=PostingsFormat(name=NestedPulsing), id=PostingsFormat(name=Direct), a_t=PostingsFormat(name=Lucene41VarGapFixedInterval), rnd_b=PostingsFormat(name=Direct)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=et, timezone=Etc/GMT+4
[junit4] 2> NOTE: Linux 3.8.0-36-generic i386/Oracle Corporation 1.7.0_60-ea (32-bit)/cpus=8,threads=85,free=324068616,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [IndexSchemaTest, TestUniqueKeyFieldResource, TestDefaultSearchFieldResource, BasicFunctionalityTest, CSVRequestHandlerTest, ClusterStateUpdateTest, DistributedSpellCheckComponentTest, HdfsSyncSliceTest, ExternalFileFieldSortTest, UpdateParamsTest, TestJmxMonitoredMap, TestConfig, CoreAdminHandlerTest, AnalysisAfterCoreReloadTest, TestSystemIdResolver, ScriptEngineTest, EchoParamsTest, TestTrie, TermsComponentTest, TestSolrDeletionPolicy1, TestElisionMultitermQuery, DocValuesMissingTest, TestPostingsSolrHighlighter, LukeRequestHandlerTest, TestSchemaVersionResource, TestRandomDVFaceting, AnalysisErrorHandlingTest, RegexBoostProcessorTest, TestRemoteStreaming, TestStressUserVersions, FastVectorHighlighterTest, TriLevelCompositeIdRoutingTest, SuggesterTest, SpellPossibilityIteratorTest, FunctionTest, CursorPagingTest, DistributedSuggestComponentTest, QueryElevationComponentTest, SpatialFilterTest, TestSolrQueryParserResource, PeerSyncTest, TestInitQParser, SliceStateUpdateTest, DateMathParserTest, TestAddFieldRealTimeGet, SoftAutoCommitTest, TestMergePolicyConfig, BadCopyFieldTest, TestSuggestSpellingConverter, SolrInfoMBeanTest, ResponseLogComponentTest, SearchHandlerTest, TestInfoStreamLogging, InfoHandlerTest, CoreAdminCreateDiscoverTest, BinaryUpdateRequestHandlerTest, TermVectorComponentTest, TestPseudoReturnFields, QueryResultKeyTest, SampleTest, TestGroupingSearch, TestPerFieldSimilarity, TestRealTimeGet, AbstractAnalyticsStatsTest, DirectUpdateHandlerOptimizeTest, DeleteShardTest, TestLMDirichletSimilarityFactory, TestCopyFieldCollectionResource, HdfsDirectoryTest, TestCoreContainer, ReturnFieldsTest, LegacyHTMLStripCharFilterTest, BasicDistributedZkTest, TestLMJelinekMercerSimilarityFactory, DOMUtilTest, TestQueryUtils, TestSolrIndexConfig, ExpressionTest, ShowFileRequestHandlerTest, TestSurroundQueryParser, DistributedDebugComponentTest, FieldMutatingUpdateProcessorTest, BlockCacheTest, TestLuceneMatchVersion, TestExtendedDismaxParser, TestCollapseQParserPlugin, TestNonNRTOpen, MBeansHandlerTest, ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=7251B54D221CEEDC -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=et -Dtests.timezone=Etc/GMT+4 -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([7251B54D221CEEDC]:0)
[junit4] Completed on J0 in 7224.51s, 1 test, 2 errors <<< FAILURES!
[...truncated 11 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:465: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:445: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1274: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 373 suites, 1606 tests, 1 suite-level error, 1 error, 35 ignored (7 assumptions)
Total time: 149 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b04 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-fcs-b129) - Build
# 9594 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9594/
Java: 32bit/jdk1.8.0-fcs-b129 -server -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testShutdown
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:39963 within 30000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:39963 within 30000 ms
at __randomizedtesting.SeedInfo.seed([D0C8D67A079EC445:33BEDFEFE0E45137]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:200)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:39963 within 30000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:222)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
... 47 more
Build Log:
[...truncated 11515 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 12655 T54 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 12656 T54 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 12658 T54 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 12663 T54 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1393351627658
[junit4] 2> 12667 T54 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 12669 T55 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 12769 T54 oasc.ZkTestServer.run start zk server on port:39963
[junit4] 2> 12822 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 83589 T58 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 70748ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 83592 T54 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 83609 T56 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446a3a93c80000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 83611 T54 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:39963 39963
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testShutdown -Dtests.seed=D0C8D67A079EC445 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_UY -Dtests.timezone=Asia/Harbin -Dtests.file.encoding=US-ASCII
[junit4] ERROR 76.3s J0 | CloudSolrServerTest.testShutdown <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:39963 within 30000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([D0C8D67A079EC445:33BEDFEFE0E45137]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:200)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:77)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:39963 within 30000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:222)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
[junit4] > ... 47 more
[junit4] 2> 89006 T54 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1393351704001
[junit4] 2> 89007 T54 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 89008 T62 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 89108 T54 oasc.ZkTestServer.run start zk server on port:58922
[junit4] 2> 89109 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 89290 T68 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b1222e name:ZooKeeperConnection Watcher:127.0.0.1:58922 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 89291 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 89293 T54 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 89315 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 89318 T70 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@247f0e name:ZooKeeperConnection Watcher:127.0.0.1:58922/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 89318 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 89324 T54 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 89328 T54 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 89332 T54 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 89335 T54 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 89341 T54 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 89341 T54 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 89347 T54 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 89347 T54 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 89351 T54 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 89352 T54 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 89352 T54 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 89353 T54 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 89353 T54 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 89354 T54 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 89354 T54 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 89355 T54 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 89355 T54 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 89361 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 89363 T72 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b29ea6 name:ZooKeeperConnection Watcher:127.0.0.1:58922/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 89363 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 89535 T54 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 89749 T54 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 89774 T54 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:35842
[junit4] 2> 89793 T54 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 89794 T54 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 89794 T54 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393351704362
[junit4] 2> 89794 T54 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393351704362/'
[junit4] 2> 89819 T54 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393351704362/solr.xml
[junit4] 2> 89828 T54 oasc.CoreContainer.<init> New CoreContainer 31647156
[junit4] 2> 89829 T54 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393351704362/]
[junit4] 2> 89829 T54 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 89830 T54 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 89830 T54 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 89831 T54 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 89831 T54 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 89832 T54 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 89832 T54 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 89833 T54 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 89833 T54 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 89844 T54 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 89845 T54 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 89845 T54 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 89846 T54 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58922/solr
[junit4] 2> 89855 T54 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 89856 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 89858 T84 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@914508 name:ZooKeeperConnection Watcher:127.0.0.1:58922 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 89859 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 89868 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 89870 T86 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6cda2c name:ZooKeeperConnection Watcher:127.0.0.1:58922/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 89871 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 89876 T54 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 89882 T54 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 89897 T54 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 89899 T54 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35842_
[junit4] 2> 89901 T54 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35842_
[junit4] 2> 89905 T54 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 89908 T54 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 89917 T54 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:35842_
[junit4] 2> 89917 T54 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 89920 T54 oasc.Overseer.start Overseer (id=91314697273540612-127.0.0.1:35842_-n_0000000000) starting
[junit4] 2> 89930 T54 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 89944 T88 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 89945 T54 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 89946 T88 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 89950 T54 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 89953 T54 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 89962 T87 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 89969 T89 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 89969 T89 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 89972 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 89972 T89 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 89975 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 89976 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:35842",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35842_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 89976 T87 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 89986 T87 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 89990 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 89995 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 90972 T89 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 90972 T89 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393351704362/collection1
[junit4] 2> 90973 T89 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 90973 T89 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 90974 T89 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 90975 T89 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 90976 T89 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393351704362/collection1/'
[junit4] 2> 91000 T89 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 91012 T89 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 91014 T89 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 91030 T89 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 91172 T89 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 91174 T89 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 91175 T89 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 91176 T89 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 91177 T89 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 91177 T89 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393351704362/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/control/data/
[junit4] 2> 91177 T89 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 91178 T89 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/control/data
[junit4] 2> 91178 T89 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/control/data/index/
[junit4] 2> 91178 T89 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 91179 T89 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/control/data/index
[junit4] 2> 91181 T89 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@13f9b91 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 91181 T89 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 91182 T89 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 91183 T89 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 91183 T89 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 91183 T89 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 91183 T89 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 91184 T89 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 91184 T89 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 91184 T89 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 91186 T89 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 91187 T89 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 91187 T89 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 91188 T89 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@13f9b91 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 91188 T89 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 91188 T89 oass.SolrIndexSearcher.<init> Opening Searcher@1f11575[collection1] main
[junit4] 2> 91189 T89 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 91190 T90 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f11575[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 91190 T54 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 91190 T93 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:35842 collection:control_collection shard:shard1
[junit4] 2> 91191 T54 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 91198 T93 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 91206 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 91208 T95 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c1fefa name:ZooKeeperConnection Watcher:127.0.0.1:58922/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 91208 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 91211 T93 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 91211 T54 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 91213 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 91214 T93 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 91215 T93 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 91215 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 91217 T54 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> ASYNC NEW_CORE C19 name=collection1 org.apache.solr.core.SolrCore@1b074b5 url=https://127.0.0.1:35842/collection1 node=127.0.0.1:35842_ C19_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:35842, core=collection1, node_name=127.0.0.1:35842_}
[junit4] 2> 91220 T93 C19 P35842 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:35842/collection1/
[junit4] 2> 91223 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 91223 T93 C19 P35842 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 91223 T93 C19 P35842 oasc.SyncStrategy.syncToMe https://127.0.0.1:35842/collection1/ has no replicas
[junit4] 2> 91224 T93 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:35842/collection1/ shard1
[junit4] 2> 91224 T93 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 91230 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 91236 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 91247 T54 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 91250 T54 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 91255 T54 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 91260 T54 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:38083
[junit4] 2> 91263 T54 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 91263 T54 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 91264 T54 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393351706211
[junit4] 2> 91271 T54 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393351706211/'
[junit4] 2> 91286 T54 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393351706211/solr.xml
[junit4] 2> 91299 T54 oasc.CoreContainer.<init> New CoreContainer 21897166
[junit4] 2> 91299 T54 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393351706211/]
[junit4] 2> 91303 T54 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 91304 T54 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 91304 T54 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 91305 T54 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 91306 T54 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 91306 T54 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 91306 T54 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 91307 T54 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 91307 T54 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 91313 T54 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 91314 T54 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 91314 T54 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 91315 T54 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58922/solr
[junit4] 2> 91315 T54 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 91316 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 91319 T106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16790f3 name:ZooKeeperConnection Watcher:127.0.0.1:58922 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 91320 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 91323 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 91324 T108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fa80fd name:ZooKeeperConnection Watcher:127.0.0.1:58922/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 91324 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 91329 T54 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 91339 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 91339 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 91339 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 91381 T93 oasc.ZkController.register We are https://127.0.0.1:35842/collection1/ and leader is https://127.0.0.1:35842/collection1/
[junit4] 2> 91382 T93 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:35842
[junit4] 2> 91382 T93 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 91382 T93 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 91382 T93 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 91383 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 91383 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 91384 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 91384 T93 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 91385 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 91386 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:35842",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35842_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 91389 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 91491 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 91491 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 91491 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 92333 T54 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38083_
[junit4] 2> 92335 T54 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38083_
[junit4] 2> 92340 T95 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 92340 T108 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 92340 T86 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 92347 T109 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 92347 T109 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 92348 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 92348 T109 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 92348 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 92348 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 92350 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 92351 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38083",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38083_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 92351 T87 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 92351 T87 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 92354 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 92459 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 92459 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 92459 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 93348 T109 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 93349 T109 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393351706211/collection1
[junit4] 2> 93349 T109 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 93351 T109 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 93351 T109 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 93354 T109 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 93354 T109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393351706211/collection1/'
[junit4] 2> 93379 T109 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 93397 T109 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 93399 T109 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 93408 T109 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 93527 T109 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 93529 T109 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 93531 T109 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 93531 T109 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 93532 T109 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 93533 T109 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393351706211/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty1/
[junit4] 2> 93533 T109 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 93534 T109 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty1
[junit4] 2> 93534 T109 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty1/index/
[junit4] 2> 93534 T109 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 93534 T109 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty1/index
[junit4] 2> 93536 T109 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@ae1af7 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 93536 T109 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 93537 T109 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 93538 T109 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 93538 T109 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 93538 T109 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 93538 T109 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 93539 T109 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 93539 T109 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 93539 T109 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 93541 T109 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 93543 T109 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 93543 T109 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 93544 T109 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@ae1af7 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 93544 T109 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 93545 T109 oass.SolrIndexSearcher.<init> Opening Searcher@f187ad[collection1] main
[junit4] 2> 93545 T110 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f187ad[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 93545 T109 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 93546 T54 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 93547 T54 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 93546 T113 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:38083 collection:collection1 shard:shard2
[junit4] 2> 93553 T113 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 93563 T113 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 93567 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93567 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93568 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93569 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 93574 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93574 T113 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 93575 T113 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C20 name=collection1 org.apache.solr.core.SolrCore@1593e1f url=https://127.0.0.1:38083/collection1 node=127.0.0.1:38083_ C20_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:38083, core=collection1, node_name=127.0.0.1:38083_}
[junit4] 2> 93575 T113 C20 P38083 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:38083/collection1/
[junit4] 2> 93575 T113 C20 P38083 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 93575 T113 C20 P38083 oasc.SyncStrategy.syncToMe https://127.0.0.1:38083/collection1/ has no replicas
[junit4] 2> 93576 T113 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:38083/collection1/ shard2
[junit4] 2> 93576 T113 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 93583 T54 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 93583 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93584 T54 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 93587 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93588 T54 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 93591 T54 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:33662
[junit4] 2> 93594 T54 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 93594 T54 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 93595 T54 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393351708543
[junit4] 2> 93595 T54 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393351708543/'
[junit4] 2> 93613 T54 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393351708543/solr.xml
[junit4] 2> 93623 T54 oasc.CoreContainer.<init> New CoreContainer 14111405
[junit4] 2> 93623 T54 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393351708543/]
[junit4] 2> 93624 T54 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 93624 T54 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 93625 T54 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 93625 T54 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 93625 T54 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 93628 T54 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 93628 T54 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 93629 T54 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 93629 T54 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 93634 T54 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 93634 T54 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 93635 T54 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 93635 T54 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58922/solr
[junit4] 2> 93635 T54 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 93636 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 93641 T124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26bb6f name:ZooKeeperConnection Watcher:127.0.0.1:58922 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 93641 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 93644 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 93646 T126 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@127c5ac name:ZooKeeperConnection Watcher:127.0.0.1:58922/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 93646 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 93651 T54 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 93692 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 93692 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 93692 T126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 93692 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 93742 T113 oasc.ZkController.register We are https://127.0.0.1:38083/collection1/ and leader is https://127.0.0.1:38083/collection1/
[junit4] 2> 93742 T113 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:38083
[junit4] 2> 93743 T113 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 93743 T113 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 93743 T113 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 93745 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93745 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93746 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93746 T113 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 93748 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 93750 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:38083",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38083_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 93755 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 93859 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 93859 T126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 93859 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 93859 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 94658 T54 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33662_
[junit4] 2> 94661 T54 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33662_
[junit4] 2> 94665 T108 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 94665 T86 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 94665 T126 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 94666 T95 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 94674 T127 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 94674 T127 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 94676 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 94676 T127 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 94676 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 94676 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 94679 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 94680 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33662",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33662_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 94681 T87 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 94681 T87 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 94686 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 94790 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 94790 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 94790 T126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 94790 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 95677 T127 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 95677 T127 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393351708543/collection1
[junit4] 2> 95677 T127 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 95679 T127 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 95679 T127 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 95681 T127 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 95682 T127 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393351708543/collection1/'
[junit4] 2> 95710 T127 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 95731 T127 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 95733 T127 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 95747 T127 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 95896 T127 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 95898 T127 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 95899 T127 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 95900 T127 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 95901 T127 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 95901 T127 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393351708543/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty2/
[junit4] 2> 95901 T127 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 95902 T127 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty2
[junit4] 2> 95902 T127 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty2/index/
[junit4] 2> 95902 T127 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 95902 T127 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty2/index
[junit4] 2> 95903 T127 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1b1617f lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 95904 T127 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 95905 T127 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 95905 T127 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 95905 T127 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 95905 T127 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 95905 T127 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 95906 T127 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 95906 T127 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 95906 T127 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 95907 T127 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 95908 T127 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 95909 T127 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 95909 T127 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1b1617f lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 95909 T127 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 95910 T127 oass.SolrIndexSearcher.<init> Opening Searcher@1d17ebd[collection1] main
[junit4] 2> 95910 T128 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d17ebd[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 95910 T127 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 95911 T54 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 95912 T54 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 95911 T131 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:33662 collection:collection1 shard:shard1
[junit4] 2> 95913 T131 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 95918 T131 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 95920 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 95920 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 95920 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 95921 T131 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 95921 T131 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C21 name=collection1 org.apache.solr.core.SolrCore@1bb5e11 url=https://127.0.0.1:33662/collection1 node=127.0.0.1:33662_ C21_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:33662, core=collection1, node_name=127.0.0.1:33662_}
[junit4] 2> 95921 T131 C21 P33662 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:33662/collection1/
[junit4] 2> 95922 T131 C21 P33662 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 95922 T131 C21 P33662 oasc.SyncStrategy.syncToMe https://127.0.0.1:33662/collection1/ has no replicas
[junit4] 2> 95922 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 95922 T131 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:33662/collection1/ shard1
[junit4] 2> 95922 T131 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 95925 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 95930 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 95933 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 95943 T54 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 95944 T54 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 95949 T54 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 95952 T54 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:46876
[junit4] 2> 95954 T54 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 95955 T54 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 95955 T54 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393351710907
[junit4] 2> 95956 T54 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393351710907/'
[junit4] 2> 95972 T54 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393351710907/solr.xml
[junit4] 2> 95990 T54 oasc.CoreContainer.<init> New CoreContainer 19293831
[junit4] 2> 95991 T54 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393351710907/]
[junit4] 2> 95992 T54 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 95992 T54 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 95993 T54 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 95993 T54 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 95993 T54 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 95994 T54 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 95994 T54 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 95995 T54 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 95995 T54 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 96000 T54 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 96001 T54 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 96001 T54 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 96001 T54 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58922/solr
[junit4] 2> 96002 T54 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 96003 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 96008 T142 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cd6c1 name:ZooKeeperConnection Watcher:127.0.0.1:58922 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 96008 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 96012 T54 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 96014 T144 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e88a47 name:ZooKeeperConnection Watcher:127.0.0.1:58922/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 96014 T54 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 96019 T54 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 96037 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 96037 T126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 96037 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 96037 T144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 96037 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 96084 T131 oasc.ZkController.register We are https://127.0.0.1:33662/collection1/ and leader is https://127.0.0.1:33662/collection1/
[junit4] 2> 96084 T131 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:33662
[junit4] 2> 96084 T131 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 96084 T131 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 96084 T131 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 96086 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 96086 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 96086 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 96086 T131 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 96087 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 96088 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:33662",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33662_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 96092 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 96196 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 96196 T144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 96196 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 96196 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 96196 T126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 97023 T54 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46876_
[junit4] 2> 97024 T54 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46876_
[junit4] 2> 97029 T86 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 97029 T126 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 97029 T144 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 97029 T95 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 97029 T108 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 97040 T145 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 97040 T145 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 97042 T145 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 97042 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 97043 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 97043 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 97045 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 97047 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:46876",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:46876_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 97047 T87 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 97047 T87 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 97052 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 97156 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 97156 T126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 97156 T144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 97156 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 97156 T86 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 98042 T145 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 98043 T145 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393351710907/collection1
[junit4] 2> 98043 T145 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 98044 T145 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 98045 T145 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 98046 T145 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 98046 T145 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393351710907/collection1/'
[junit4] 2> 98072 T145 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 98092 T145 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 98095 T145 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 98106 T145 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 98272 T145 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 98274 T145 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 98275 T145 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 98275 T145 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 98276 T145 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 98276 T145 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393351710907/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3/
[junit4] 2> 98276 T145 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 98277 T145 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3
[junit4] 2> 98277 T145 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3/index/
[junit4] 2> 98277 T145 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 98278 T145 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3/index
[junit4] 2> 98279 T145 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@8e6008 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3/index),segFN=segments_1,generation=1}
[junit4] 2> 98279 T145 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 98280 T145 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 98280 T145 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 98281 T145 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 98281 T145 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 98281 T145 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 98281 T145 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 98281 T145 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 98282 T145 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 98284 T145 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 98285 T145 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 98286 T145 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 98286 T145 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@8e6008 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3/index),segFN=segments_1,generation=1}
[junit4] 2> 98287 T145 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 98287 T145 oass.SolrIndexSearcher.<init> Opening Searcher@1c88cda[collection1] main
[junit4] 2> 98288 T146 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c88cda[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 98288 T145 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 98289 T54 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 98289 T54 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 98289 T149 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:46876 collection:collection1 shard:shard2
[junit4] 2> 98295 T149 oasc.ZkController.register We are https://127.0.0.1:46876/collection1/ and leader is https://127.0.0.1:38083/collection1/
[junit4] 2> 98295 T149 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:46876
[junit4] 2> 98295 T149 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 98295 T149 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C22 name=collection1 org.apache.solr.core.SolrCore@4914d8 url=https://127.0.0.1:46876/collection1 node=127.0.0.1:46876_ C22_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:46876, core=collection1, node_name=127.0.0.1:46876_}
[junit4] 2> 98300 T150 C22 P46876 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 98301 T150 C22 P46876 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 98301 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 98302 T150 C22 P46876 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 98302 T150 C22 P46876 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 98303 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 98303 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 98303 T86 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 98304 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 98305 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "base_url":"https://127.0.0.1:46876",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:46876_",
[junit4] 2> "shard":"
[...truncated too long message...]
:0.0.0.0/0.0.0.0:0
[junit4] 2> 114554 T163 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 114554 T163 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@11b97b1
[junit4] 2> 114555 T163 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=6,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=2,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=5}
[junit4] 2> 114555 T163 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 114555 T163 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 114555 T163 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 114556 T54 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 114558 T144 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 114558 T163 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 114558 T163 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 114558 T144 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4914d8
[junit4] 2> 114558 T163 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty4;done=false>>]
[junit4] 2> 114559 T163 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty4
[junit4] 2> 114559 T163 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty4/index;done=false>>]
[junit4] 2> 114559 T163 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty4/index
[junit4] 2> 114559 T163 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 114560 T163 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 114564 T144 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=6,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=2,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=5}
[junit4] 2> 114568 T144 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 114568 T144 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 114569 T144 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 114570 T144 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 114570 T144 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 114570 T144 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3/index;done=false>>]
[junit4] 2> 114570 T144 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3/index
[junit4] 2> 114571 T144 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3;done=false>>]
[junit4] 2> 114571 T144 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393351704001/jetty3
[junit4] 2> 114571 T144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 114571 T144 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 114571 T144 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 114572 T144 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Lucene46: {_version_=MockFixedIntBlock(blockSize=1404), multiDefault=PostingsFormat(name=Memory doPackFST= true), a_t=MockFixedIntBlock(blockSize=1404), intDefault=MockFixedIntBlock(blockSize=1404), range_facet_sl=PostingsFormat(name=Lucene41WithOrds), range_facet_si=PostingsFormat(name=Memory doPackFST= true), id=MockVariableIntBlock(baseBlockSize=16), text=PostingsFormat(name=Lucene41WithOrds), range_facet_l=MockFixedIntBlock(blockSize=1404), timestamp=MockFixedIntBlock(blockSize=1404)}, docValues:{}, sim=DefaultSimilarity, locale=es_UY, timezone=Asia/Harbin
[junit4] 2> NOTE: Linux 3.8.0-36-generic i386/Oracle Corporation 1.8.0 (32-bit)/cpus=8,threads=1,free=35818344,total=64880640
[junit4] 2> NOTE: All tests run in this JVM: [SolrExceptionTest, SolrPingTest, TestXMLEscaping, SolrParamTest, ClientUtilsTest, SolrExampleEmbeddedTest, AnlysisResponseBaseTest, LargeVolumeEmbeddedTest, ModifiableSolrParamsTest, CloudSolrServerTest]
[junit4] Completed on J0 in 102.93s, 2 tests, 1 error <<< FAILURES!
[...truncated 64 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:465: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:445: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1274: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 48 suites, 280 tests, 1 error
Total time: 55 minutes 5 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-fcs-b129 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure