You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/03/27 06:16:28 UTC
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0) - Build # 9917 -
Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9917/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([A6004BBA0211885B:27E6C5A2754EE867]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.doTest(CloudSolrServerTest.java:256)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
at java.lang.Thread.run(Thread.java:744)
Build Log:
[...truncated 11846 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 22293 T115 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 22294 T115 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /dod/lj
[junit4] 2> 22297 T115 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 22301 T115 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395897312881
[junit4] 2> 22305 T115 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 22308 T116 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 22496 T115 oasc.ZkTestServer.run start zk server on port:53230
[junit4] 2> 22543 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 22567 T117 oazs.NIOServerCnxn.doIO WARN Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4] 2> 28573 T119 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 4571ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 28586 T122 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1168a895 name:ZooKeeperConnection Watcher:127.0.0.1:53230 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 28586 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 28588 T115 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 28614 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 28617 T124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b5c8090 name:ZooKeeperConnection Watcher:127.0.0.1:53230/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 28617 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 28620 T115 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 28625 T115 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 28628 T115 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 28631 T115 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 28635 T115 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 28636 T115 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 28641 T115 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 28641 T115 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 28647 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 28647 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 28647 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 28648 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 28648 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 28648 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 28648 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 28649 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 28649 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 28652 T117 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14501f6a6d60001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 28658 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 28660 T125 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 28660 T125 oaz.ClientCnxn$SendThread.run WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable
[junit4] 2> at sun.nio.ch.Net.connect0(Native Method)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:435)
[junit4] 2> at sun.nio.ch.Net.connect(Net.java:427)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:643)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 29763 T125 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
[junit4] 2> 29865 T115 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 29867 T115 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53230 53230
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 29987 T115 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1395897320566
[junit4] 2> 29988 T115 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 29988 T127 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 30088 T115 oasc.ZkTestServer.run start zk server on port:40209
[junit4] 2> 30090 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 30105 T133 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@330aaf60 name:ZooKeeperConnection Watcher:127.0.0.1:40209 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 30105 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 30105 T115 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 30112 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 30117 T135 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4863f216 name:ZooKeeperConnection Watcher:127.0.0.1:40209/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 30117 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 30117 T115 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 30123 T115 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 30126 T115 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 30130 T115 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 30132 T115 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 30132 T115 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 30136 T115 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 30136 T115 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 30140 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 30140 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 30140 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 30140 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 30140 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 30141 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 30141 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 30141 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 30141 T115 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 30235 T115 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 30238 T115 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47786
[junit4] 2> 30239 T115 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 30239 T115 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 30239 T115 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395897320722
[junit4] 2> 30240 T115 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395897320722/'
[junit4] 2> 30250 T115 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395897320722/solr.xml
[junit4] 2> 30267 T115 oasc.CoreContainer.<init> New CoreContainer 2145030369
[junit4] 2> 30267 T115 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395897320722/]
[junit4] 2> 30268 T115 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 30269 T115 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 30269 T115 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 30269 T115 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 30269 T115 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 30270 T115 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 30270 T115 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 30270 T115 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 30271 T115 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 30273 T115 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 30274 T115 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 30274 T115 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 30275 T115 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40209/solr
[junit4] 2> 30288 T115 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 30289 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 30292 T146 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c1e2f12 name:ZooKeeperConnection Watcher:127.0.0.1:40209 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 30293 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 30300 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 30302 T148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@683ac5d1 name:ZooKeeperConnection Watcher:127.0.0.1:40209/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 30302 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 30309 T115 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 30325 T115 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 30341 T115 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 30347 T115 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 30350 T115 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 30359 T115 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 30361 T115 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47786_dod%2Flj
[junit4] 2> 30364 T115 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47786_dod%2Flj
[junit4] 2> 30368 T115 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 30371 T115 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 30473 T115 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:47786_dod%2Flj
[junit4] 2> 30474 T115 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 30478 T115 oasc.Overseer.start Overseer (id=91481526800744451-127.0.0.1:47786_dod%2Flj-n_0000000000) starting
[junit4] 2> 30491 T115 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 30538 T150 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 30538 T115 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 30543 T115 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 30545 T115 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 30550 T150 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 30560 T149 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 30571 T151 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 30573 T151 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 30579 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30579 T151 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 30583 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 30584 T149 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:47786/dod/lj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47786_dod%2Flj",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 30584 T149 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 30595 T149 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 30604 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30612 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 31580 T151 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 31580 T151 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 31581 T151 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 31581 T151 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 31583 T151 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 31583 T151 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395897320722/collection1/'
[junit4] 2> 31619 T151 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 31634 T151 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 31636 T151 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 31648 T151 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 31674 T151 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 31680 T151 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableLongField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 31686 T151 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableFloatField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 31693 T151 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableDoubleField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 31916 T151 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 31917 T151 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 31919 T151 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 31920 T151 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 31923 T151 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 31923 T151 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 31923 T151 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1395897320722/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/control/data/
[junit4] 2> 31923 T151 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 31924 T151 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/control/data
[junit4] 2> 31924 T151 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/control/data/index/
[junit4] 2> 31924 T151 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 31924 T151 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/control/data/index
[junit4] 2> 31926 T151 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1b136c51 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 31926 T151 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 31928 T151 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 31928 T151 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 31928 T151 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 31928 T151 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 31929 T151 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 31929 T151 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 31929 T151 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 31929 T151 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 31931 T151 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 31933 T151 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 31933 T151 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 31934 T151 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1b136c51 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 31934 T151 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 31935 T151 oass.SolrIndexSearcher.<init> Opening Searcher@750dccfe[collection1] main
[junit4] 2> 31935 T151 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 31936 T151 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 31937 T151 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 31940 T151 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 31940 T151 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 31941 T151 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 31942 T151 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 31942 T151 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 31942 T151 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 31942 T151 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 31943 T151 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 31944 T152 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@750dccfe[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 31949 T155 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47786/dod/lj collection:control_collection shard:shard1
[junit4] 2> 31950 T115 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 31950 T115 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 31953 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 31960 T155 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 31960 T157 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3651e89c name:ZooKeeperConnection Watcher:127.0.0.1:40209/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 31960 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 31962 T115 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 31967 T115 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 31968 T155 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 31971 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 31972 T155 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 31972 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 31972 T155 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C24 name=collection1 org.apache.solr.core.SolrCore@58b9e639 url=http://127.0.0.1:47786/dod/lj/collection1 node=127.0.0.1:47786_dod%2Flj C24_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:47786/dod/lj, core=collection1, node_name=127.0.0.1:47786_dod%2Flj}
[junit4] 2> 31974 T155 C24 P47786 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47786/dod/lj/collection1/
[junit4] 2> 31976 T155 C24 P47786 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 31976 T155 C24 P47786 oasc.SyncStrategy.syncToMe http://127.0.0.1:47786/dod/lj/collection1/ has no replicas
[junit4] 2> 31976 T155 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47786/dod/lj/collection1/ shard1
[junit4] 2> 31979 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 31980 T155 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 32016 T115 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 32016 T115 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 32019 T115 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35420
[junit4] 2> 32019 T115 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 32019 T115 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 32020 T115 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395897322546
[junit4] 2> 32020 T115 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395897322546/'
[junit4] 2> 32032 T115 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395897322546/solr.xml
[junit4] 2> 32056 T115 oasc.CoreContainer.<init> New CoreContainer 1463081863
[junit4] 2> 32057 T115 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395897322546/]
[junit4] 2> 32059 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32063 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32064 T115 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 32065 T115 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 32065 T115 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 32065 T115 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 32065 T115 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 32066 T115 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 32066 T115 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 32066 T115 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 32066 T115 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 32067 T115 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 32067 T115 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 32067 T115 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 32067 T115 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40209/solr
[junit4] 2> 32068 T115 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 32071 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 32073 T168 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72b991a4 name:ZooKeeperConnection Watcher:127.0.0.1:40209 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 32073 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 32076 T128 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14501f6c4780005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 32077 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 32081 T170 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7678a10d name:ZooKeeperConnection Watcher:127.0.0.1:40209/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 32081 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 32088 T115 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 32166 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 32166 T157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 32166 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 32211 T155 oasc.ZkController.register We are http://127.0.0.1:47786/dod/lj/collection1/ and leader is http://127.0.0.1:47786/dod/lj/collection1/
[junit4] 2> 32211 T155 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47786/dod/lj
[junit4] 2> 32211 T155 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 32211 T155 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 32211 T155 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 32212 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32213 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32213 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32213 T155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 32214 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 32215 T149 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:47786/dod/lj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47786_dod%2Flj",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 32219 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32322 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 32322 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 32322 T157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 33092 T115 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35420_dod%2Flj
[junit4] 2> 33094 T115 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35420_dod%2Flj
[junit4] 2> 33099 T170 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 33099 T148 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 33099 T157 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 33111 T171 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 33111 T171 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 33116 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 33116 T171 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 33116 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 33117 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 33119 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 33121 T149 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35420/dod/lj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35420_dod%2Flj",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 33122 T149 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 33122 T149 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 33128 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 33231 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 33231 T157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 33231 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 34116 T171 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 34117 T171 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 34117 T171 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 34118 T171 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 34119 T171 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 34119 T171 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395897322546/collection1/'
[junit4] 2> 34139 T171 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 34158 T171 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 34159 T171 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 34167 T171 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 34291 T171 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 34293 T171 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 34295 T171 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 34295 T171 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 34297 T171 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 34297 T171 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 34298 T171 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1395897322546/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty1/
[junit4] 2> 34298 T171 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 34299 T171 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty1
[junit4] 2> 34299 T171 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty1/index/
[junit4] 2> 34299 T171 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 34299 T171 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty1/index
[junit4] 2> 34301 T171 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@40db8c10 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 34301 T171 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 34310 T171 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 34311 T171 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 34311 T171 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 34311 T171 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 34311 T171 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 34311 T171 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 34312 T171 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 34312 T171 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 34314 T171 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 34321 T171 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 34322 T171 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 34323 T171 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@40db8c10 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 34323 T171 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 34324 T171 oass.SolrIndexSearcher.<init> Opening Searcher@47f8c7ed[collection1] main
[junit4] 2> 34324 T171 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 34326 T171 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 34326 T171 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 34327 T171 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 34327 T171 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 34328 T171 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 34329 T171 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 34329 T171 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 34329 T171 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 34329 T171 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 34330 T172 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@47f8c7ed[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 34330 T171 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 34332 T175 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35420/dod/lj collection:collection1 shard:shard2
[junit4] 2> 34332 T115 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 34333 T115 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 34334 T175 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 34358 T175 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 34360 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34360 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34361 T175 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 34361 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34361 T175 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 34362 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C25 name=collection1 org.apache.solr.core.SolrCore@13c6f609 url=http://127.0.0.1:35420/dod/lj/collection1 node=127.0.0.1:35420_dod%2Flj C25_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:35420/dod/lj, core=collection1, node_name=127.0.0.1:35420_dod%2Flj}
[junit4] 2> 34363 T175 C25 P35420 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35420/dod/lj/collection1/
[junit4] 2> 34364 T175 C25 P35420 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 34365 T175 C25 P35420 oasc.SyncStrategy.syncToMe http://127.0.0.1:35420/dod/lj/collection1/ has no replicas
[junit4] 2> 34365 T175 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35420/dod/lj/collection1/ shard2
[junit4] 2> 34365 T175 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 34368 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34385 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34404 T115 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 34405 T115 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 34408 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34413 T115 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50274
[junit4] 2> 34414 T115 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 34415 T115 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 34415 T115 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395897324913
[junit4] 2> 34415 T115 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395897324913/'
[junit4] 2> 34428 T115 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395897324913/solr.xml
[junit4] 2> 34442 T115 oasc.CoreContainer.<init> New CoreContainer 2002747522
[junit4] 2> 34443 T115 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395897324913/]
[junit4] 2> 34443 T115 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 34443 T115 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 34444 T115 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 34444 T115 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 34444 T115 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 34445 T115 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 34445 T115 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 34445 T115 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 34446 T115 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 34446 T115 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 34448 T115 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 34449 T115 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 34449 T115 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40209/solr
[junit4] 2> 34450 T115 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 34450 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 34453 T186 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67aeb1bd name:ZooKeeperConnection Watcher:127.0.0.1:40209 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 34454 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 34456 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 34514 T188 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b3dd0f9 name:ZooKeeperConnection Watcher:127.0.0.1:40209/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 34515 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 34516 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 34516 T157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 34516 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 34525 T115 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 34536 T175 oasc.ZkController.register We are http://127.0.0.1:35420/dod/lj/collection1/ and leader is http://127.0.0.1:35420/dod/lj/collection1/
[junit4] 2> 34536 T175 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35420/dod/lj
[junit4] 2> 34536 T175 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 34537 T175 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 34537 T175 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 34539 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34539 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34539 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34540 T175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 34541 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 34541 T149 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:35420/dod/lj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35420_dod%2Flj",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 34544 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34648 T157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 34648 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 34648 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 34648 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 35528 T115 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50274_dod%2Flj
[junit4] 2> 35529 T115 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50274_dod%2Flj
[junit4] 2> 35531 T157 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 35531 T170 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 35532 T188 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 35532 T148 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 35536 T189 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 35537 T189 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 35539 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 35539 T189 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 35539 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 35540 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 35541 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 35542 T149 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:50274/dod/lj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50274_dod%2Flj",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 35542 T149 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 35543 T149 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 35545 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 35647 T157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 35647 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 35647 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 35647 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 36540 T189 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 36540 T189 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 36541 T189 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 36541 T189 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 36545 T189 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 36546 T189 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395897324913/collection1/'
[junit4] 2> 36569 T189 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 36579 T189 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 36581 T189 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 36586 T189 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 36759 T189 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 36761 T189 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 36763 T189 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 36763 T189 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 36764 T189 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 36764 T189 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 36781 T189 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1395897324913/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2/
[junit4] 2> 36781 T189 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 36782 T189 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2
[junit4] 2> 36783 T189 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2/index/
[junit4] 2> 36783 T189 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 36783 T189 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2/index
[junit4] 2> 36784 T189 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@23c6d328 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 36785 T189 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 36792 T189 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 36792 T189 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 36792 T189 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 36792 T189 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 36793 T189 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 36793 T189 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 36793 T189 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 36793 T189 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 36795 T189 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 36797 T189 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 36797 T189 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 36798 T189 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@23c6d328 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 36798 T189 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 36799 T189 oass.SolrIndexSearcher.<init> Opening Searcher@44386f47[collection1] main
[junit4] 2> 36799 T189 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 36800 T189 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 36800 T189 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 36801 T189 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 36801 T189 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 36801 T189 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 36802 T189 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 36802 T189 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 36803 T189 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 36803 T189 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 36804 T190 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44386f47[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 36804 T189 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 36806 T115 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 36806 T115 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 36815 T193 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50274/dod/lj collection:collection1 shard:shard1
[junit4] 2> 36817 T193 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 36830 T193 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 36837 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 36837 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 36838 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 36839 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 36840 T193 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 36840 T193 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C26 name=collection1 org.apache.solr.core.SolrCore@40445f07 url=http://127.0.0.1:50274/dod/lj/collection1 node=127.0.0.1:50274_dod%2Flj C26_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:50274/dod/lj, core=collection1, node_name=127.0.0.1:50274_dod%2Flj}
[junit4] 2> 36840 T193 C26 P50274 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50274/dod/lj/collection1/
[junit4] 2> 36841 T193 C26 P50274 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 36841 T193 C26 P50274 oasc.SyncStrategy.syncToMe http://127.0.0.1:50274/dod/lj/collection1/ has no replicas
[junit4] 2> 36841 T193 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50274/dod/lj/collection1/ shard1
[junit4] 2> 36841 T193 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 36842 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 36847 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 36892 T115 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 36893 T115 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 36906 T115 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57030
[junit4] 2> 36907 T115 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 36908 T115 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 36908 T115 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395897327386
[junit4] 2> 36908 T115 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395897327386/'
[junit4] 2> 36924 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 36924 T115 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395897327386/solr.xml
[junit4] 2> 36938 T115 oasc.CoreContainer.<init> New CoreContainer 1481273808
[junit4] 2> 36938 T115 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1395897327386/]
[junit4] 2> 36939 T115 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 36939 T115 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 36939 T115 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 36940 T115 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 36940 T115 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 36940 T115 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 36941 T115 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 36941 T115 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 36941 T115 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 36942 T115 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 36942 T115 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 36943 T115 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 36943 T115 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40209/solr
[junit4] 2> 36943 T115 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 36944 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 36955 T204 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19aa324a name:ZooKeeperConnection Watcher:127.0.0.1:40209 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 36956 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 36958 T115 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 36976 T206 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b920302 name:ZooKeeperConnection Watcher:127.0.0.1:40209/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 36988 T115 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 37002 T115 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 37026 T157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 37026 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 37026 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 37026 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 37027 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 37057 T193 oasc.ZkController.register We are http://127.0.0.1:50274/dod/lj/collection1/ and leader is http://127.0.0.1:50274/dod/lj/collection1/
[junit4] 2> 37057 T193 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50274/dod/lj
[junit4] 2> 37057 T193 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 37057 T193 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 37057 T193 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 37059 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 37059 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 37059 T193 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 37060 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 37061 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 37062 T149 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:50274/dod/lj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50274_dod%2Flj",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 37064 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 37168 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 37169 T157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 37169 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 37168 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 37169 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 38005 T115 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57030_dod%2Flj
[junit4] 2> 38007 T115 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57030_dod%2Flj
[junit4] 2> 38010 T206 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 38010 T170 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 38010 T157 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 38010 T148 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 38010 T188 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 38026 T207 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 38027 T207 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 38029 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 38029 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 38029 T207 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 38031 T149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 38029 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 38033 T149 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:57030/dod/lj",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:57030_dod%2Flj",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 38033 T149 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 38033 T149 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 38039 T148 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 38143 T157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 38143 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 38143 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 38143 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 38143 T148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 39035 T207 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 39035 T207 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 39036 T207 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 39036 T207 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 39036 T207 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 39036 T207 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../
[...truncated too long message...]
1:50274_dod%2Flj",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 45947 T115 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 45948 T115 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2;done=false>>]
[junit4] 2> 45948 T115 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2
[junit4] 2> 45948 T115 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2/index;done=false>>]
[junit4] 2> 45948 T115 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty2/index
[junit4] 2> 45950 T188 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 45951 T188 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 45951 T128 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14501f6c4780008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 45973 T115 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/dod/lj,null}
[junit4] 2> 46024 T115 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 57030
[junit4] 2> 46024 T115 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1481273808
[junit4] 2> 46024 T115 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 46025 T115 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 46051 T115 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/dod/lj,null}
[junit4] 2> 46104 T115 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 46105 T115 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40209 40209
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testDistribSearch -Dtests.seed=A6004BBA0211885B -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hi -Dtests.timezone=Etc/Greenwich -Dtests.file.encoding=UTF-8
[junit4] FAILURE 16.3s J0 | CloudSolrServerTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([A6004BBA0211885B:27E6C5A2754EE867]:0)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.doTest(CloudSolrServerTest.java:256)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 46315 T115 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 47363 T170 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 48371 T206 C33 P57030 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 48376 T206 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 48377 T206 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@38d5fd08
[junit4] 2> 48377 T206 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=354,transaction_logs_total_number=4}
[junit4] 2> 48377 T206 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 48382 T206 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 48382 T206 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 48383 T206 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 48384 T206 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 48384 T206 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty3/index;done=false>>]
[junit4] 2> 48384 T206 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty3/index
[junit4] 2> 48385 T206 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty3;done=false>>]
[junit4] 2> 48385 T206 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1395897320566/jetty3
[junit4] 2> 48385 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 48385 T206 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 48385 T206 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 48386 T206 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=46), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=46)), sim=DefaultSimilarity, locale=hi, timezone=Etc/Greenwich
[junit4] 2> NOTE: Linux 3.8.0-37-generic amd64/Oracle Corporation 1.8.0 (64-bit)/cpus=8,threads=1,free=125086608,total=189792256
[junit4] 2> NOTE: All tests run in this JVM: [TestXMLEscaping, SolrParamTest, TestCoreAdmin, TestJavaBinCodec, SolrExceptionTest, JettyWebappTest, TestDocumentObjectBinder, LargeVolumeEmbeddedTest, ClientUtilsTest, SolrExampleXMLTest, ModifiableSolrParamsTest, NoOpResponseParserTest, CloudSolrServerTest]
[junit4] Completed on J0 in 27.05s, 2 tests, 1 failure <<< FAILURES!
[...truncated 79 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 49 suites, 284 tests, 1 failure
Total time: 61 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure