You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/02/08 05:35:18 UTC
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_60-ea-b04) - Build
# 9295 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9295/
Java: 32bit/jdk1.7.0_60-ea-b04 -client -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testShutdown
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:48862 within 30000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:48862 within 30000 ms
at __randomizedtesting.SeedInfo.seed([A59D61D031D9323F:46EB6845D6A3A74D]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:198)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:48862 within 30000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:237)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
... 47 more
Build Log:
[...truncated 11631 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> log4j:WARN No such property [conversionPattern] in org.apache.solr.util.SolrLogLayout.
[junit4] 2> 558 T9 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_/
[junit4] 2> 564 T9 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 585 T9 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1391834021934
[junit4] 2> 588 T9 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 591 T10 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 690 T9 oasc.ZkTestServer.run start zk server on port:48862
[junit4] 2> 727 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 36047 T13 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 35303ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 36063 T11 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1440fc5c06a0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 36077 T11 oazs.NIOServerCnxnFactory.run WARN Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 36123 T9 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 36125 T9 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48862 48862
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testShutdown -Dtests.seed=A59D61D031D9323F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ca_ES -Dtests.timezone=Europe/Amsterdam -Dtests.file.encoding=US-ASCII
[junit4] ERROR 35.8s J0 | CloudSolrServerTest.testShutdown <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:48862 within 30000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([A59D61D031D9323F:46EB6845D6A3A74D]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:198)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:80)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:48862 within 30000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:237)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
[junit4] > ... 47 more
[junit4] 2> 36342 T9 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1391834057691
[junit4] 2> 36343 T9 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 36344 T17 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 36444 T9 oasc.ZkTestServer.run start zk server on port:37513
[junit4] 2> 36446 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 36495 T23 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4fcb6c name:ZooKeeperConnection Watcher:127.0.0.1:37513 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 36496 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 36497 T9 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 36521 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 36523 T25 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12e372a name:ZooKeeperConnection Watcher:127.0.0.1:37513/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 36523 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 36532 T9 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 36537 T9 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 36541 T9 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 36545 T9 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 36548 T9 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 36560 T9 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 36566 T9 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 36567 T9 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 36571 T9 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 36571 T9 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 36572 T9 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 36572 T9 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 36573 T9 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 36573 T9 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 36574 T9 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 36574 T9 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 36575 T9 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 36776 T9 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 36863 T9 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:39104
[junit4] 2> 36881 T9 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 36909 T9 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 36909 T9 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1391834057925
[junit4] 2> 36910 T9 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1391834057925/'
[junit4] 2> 37026 T9 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1391834057925/solr.xml
[junit4] 2> 37173 T9 oasc.CoreContainer.<init> New CoreContainer 26867611
[junit4] 2> 37174 T9 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1391834057925/]
[junit4] 2> 37192 T9 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 37193 T9 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 37193 T9 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 37194 T9 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 37194 T9 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 37195 T9 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 37195 T9 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 37195 T9 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 37196 T9 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 37364 T9 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 37366 T9 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 37368 T9 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 37368 T9 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37513/solr
[junit4] 2> 37377 T9 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 37378 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 37382 T37 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d335ea name:ZooKeeperConnection Watcher:127.0.0.1:37513 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 37382 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 37387 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 37388 T39 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44b53c name:ZooKeeperConnection Watcher:127.0.0.1:37513/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 37388 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 37393 T9 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 37400 T9 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 37407 T9 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 37409 T9 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39104__
[junit4] 2> 37412 T9 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39104__
[junit4] 2> 37417 T9 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 37420 T9 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 37428 T9 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:39104__
[junit4] 2> 37429 T9 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 37432 T9 oasc.Overseer.start Overseer (id=91215236805033987-127.0.0.1:39104__-n_0000000000) starting
[junit4] 2> 37443 T9 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 37474 T41 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 37475 T9 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 37478 T9 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 37480 T9 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 37482 T41 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 37495 T40 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 37584 T42 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 37584 T42 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 37587 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 37587 T42 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 37590 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 37591 T40 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39104/_",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:39104__",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 37591 T40 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 37598 T40 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 37602 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 37605 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 38588 T42 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 38589 T42 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1391834057925/collection1
[junit4] 2> 38589 T42 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 38591 T42 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 38592 T42 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 38595 T42 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 38596 T42 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1391834057925/collection1/'
[junit4] 2> 38678 T42 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 38749 T42 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 38756 T42 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 38810 T42 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 38878 T42 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 38884 T42 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableLongField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 38890 T42 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableFloatField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 38896 T42 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableDoubleField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 39261 T42 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 39264 T42 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 39268 T42 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 39268 T42 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 39305 T42 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 39314 T42 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1391834057925/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/control/data/
[junit4] 2> 39316 T42 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 39337 T42 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/control/data
[junit4] 2> 39338 T42 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/control/data/index/
[junit4] 2> 39338 T42 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 39347 T42 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/control/data/index
[junit4] 2> 39388 T42 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@c4b126 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 39389 T42 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 39557 T42 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 39573 T42 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 39573 T42 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 39574 T42 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 39651 T42 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 39658 T42 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 39659 T42 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 39663 T42 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 39690 T42 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 39713 T42 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 39714 T42 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 39716 T42 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@c4b126 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 39716 T42 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 39728 T42 oass.SolrIndexSearcher.<init> Opening Searcher@1a936e2[collection1] main
[junit4] 2> 39761 T42 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 39761 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a936e2[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 39764 T9 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 39764 T9 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 39764 T46 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39104/_ collection:control_collection shard:shard1
[junit4] 2> 39771 T46 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 39779 T46 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 39781 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 39782 T46 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 39782 T46 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 39783 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@1fb28cf url=http://127.0.0.1:39104/_/collection1 node=127.0.0.1:39104__ C0_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:39104/_, core=collection1, node_name=127.0.0.1:39104__}
[junit4] 2> 39785 T46 C0 P39104 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39104/_/collection1/
[junit4] 2> 39786 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 39786 T46 C0 P39104 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 39787 T46 C0 P39104 oasc.SyncStrategy.syncToMe http://127.0.0.1:39104/_/collection1/ has no replicas
[junit4] 2> 39787 T46 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39104/_/collection1/ shard1
[junit4] 2> 39788 T46 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 39788 T48 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19638b9 name:ZooKeeperConnection Watcher:127.0.0.1:37513/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 39788 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 39789 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 39790 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 39791 T9 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 39799 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 39800 T9 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 39801 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 39804 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 39828 T9 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 39829 T9 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 39831 T9 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48684
[junit4] 2> 39832 T9 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 39832 T9 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 39832 T9 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1391834061148
[junit4] 2> 39833 T9 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1391834061148/'
[junit4] 2> 39846 T9 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1391834061148/solr.xml
[junit4] 2> 39879 T9 oasc.CoreContainer.<init> New CoreContainer 26098289
[junit4] 2> 39880 T9 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1391834061148/]
[junit4] 2> 39881 T9 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 39881 T9 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 39881 T9 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 39882 T9 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 39882 T9 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 39882 T9 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 39883 T9 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 39883 T9 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 39883 T9 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 39886 T9 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 39886 T9 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 39887 T9 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 39887 T9 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37513/solr
[junit4] 2> 39888 T9 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 39889 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 39890 T59 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@388429 name:ZooKeeperConnection Watcher:127.0.0.1:37513 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 39891 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 39893 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 39894 T61 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e23ec name:ZooKeeperConnection Watcher:127.0.0.1:37513/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 39895 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 39899 T9 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 39907 T48 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 39907 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 39908 T61 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 39950 T46 oasc.ZkController.register We are http://127.0.0.1:39104/_/collection1/ and leader is http://127.0.0.1:39104/_/collection1/
[junit4] 2> 39952 T46 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39104/_
[junit4] 2> 39952 T46 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 39952 T46 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 39953 T46 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 39954 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 39954 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 39955 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 39956 T46 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 39958 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 39959 T40 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:39104/_",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:39104__",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 39963 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 40066 T61 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 40066 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 40066 T48 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 40903 T9 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48684__
[junit4] 2> 40905 T9 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48684__
[junit4] 2> 40908 T48 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 40908 T39 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 40909 T61 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 40917 T62 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 40917 T62 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 40919 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 40919 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 40919 T62 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 40920 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 40922 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 40923 T40 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48684/_",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:48684__",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 40923 T40 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 40923 T40 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 40956 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 41059 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 41059 T61 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 41059 T48 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 41920 T62 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 41921 T62 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1391834061148/collection1
[junit4] 2> 41921 T62 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 41922 T62 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 41922 T62 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 41923 T62 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 41923 T62 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1391834061148/collection1/'
[junit4] 2> 41954 T62 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 41988 T62 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 41990 T62 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 42020 T62 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 42290 T62 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 42293 T62 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 42295 T62 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 42296 T62 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 42297 T62 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 42297 T62 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1391834061148/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty1/
[junit4] 2> 42298 T62 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 42298 T62 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty1
[junit4] 2> 42299 T62 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty1/index/
[junit4] 2> 42299 T62 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 42299 T62 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty1/index
[junit4] 2> 42301 T62 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@57962c lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 42301 T62 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 42302 T62 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 42302 T62 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 42303 T62 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 42303 T62 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 42303 T62 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 42304 T62 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 42304 T62 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 42304 T62 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 42311 T62 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 42313 T62 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 42313 T62 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 42314 T62 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@57962c lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 42314 T62 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 42315 T62 oass.SolrIndexSearcher.<init> Opening Searcher@1135b5[collection1] main
[junit4] 2> 42316 T63 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1135b5[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 42316 T62 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 42317 T9 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 42317 T9 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 42317 T66 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:48684/_ collection:collection1 shard:shard1
[junit4] 2> 42319 T66 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 42328 T66 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 42331 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42331 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42331 T66 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 42331 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42332 T66 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@7f1f06 url=http://127.0.0.1:48684/_/collection1 node=127.0.0.1:48684__ C1_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:48684/_, core=collection1, node_name=127.0.0.1:48684__}
[junit4] 2> 42332 T66 C1 P48684 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:48684/_/collection1/
[junit4] 2> 42333 T66 C1 P48684 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 42333 T66 C1 P48684 oasc.SyncStrategy.syncToMe http://127.0.0.1:48684/_/collection1/ has no replicas
[junit4] 2> 42334 T66 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:48684/_/collection1/ shard1
[junit4] 2> 42334 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 42334 T66 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 42339 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42346 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42350 T9 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 42351 T9 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 42351 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42353 T9 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42988
[junit4] 2> 42354 T9 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 42354 T9 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 42355 T9 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1391834063666
[junit4] 2> 42355 T9 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1391834063666/'
[junit4] 2> 42368 T9 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1391834063666/solr.xml
[junit4] 2> 42399 T9 oasc.CoreContainer.<init> New CoreContainer 21035993
[junit4] 2> 42399 T9 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1391834063666/]
[junit4] 2> 42400 T9 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 42400 T9 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 42401 T9 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 42401 T9 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 42401 T9 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 42402 T9 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 42402 T9 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 42402 T9 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 42402 T9 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 42405 T9 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 42406 T9 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 42406 T9 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 42406 T9 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37513/solr
[junit4] 2> 42407 T9 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 42407 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 42409 T77 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15679be name:ZooKeeperConnection Watcher:127.0.0.1:37513 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 42410 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 42412 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 42414 T79 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10540b7 name:ZooKeeperConnection Watcher:127.0.0.1:37513/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 42414 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 42418 T9 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 42456 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 42456 T61 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 42456 T48 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 42456 T79 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 42498 T66 oasc.ZkController.register We are http://127.0.0.1:48684/_/collection1/ and leader is http://127.0.0.1:48684/_/collection1/
[junit4] 2> 42498 T66 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48684/_
[junit4] 2> 42498 T66 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 42499 T66 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 42499 T66 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 42500 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42500 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42501 T66 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 42501 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42502 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 42503 T40 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:48684/_",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:48684__",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 42506 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 42610 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 42610 T48 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 42610 T61 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 42610 T79 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 43422 T9 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42988__
[junit4] 2> 43423 T9 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42988__
[junit4] 2> 43426 T39 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 43427 T61 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 43427 T79 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 43426 T48 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 43436 T80 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 43437 T80 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 43438 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 43438 T80 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 43438 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 43439 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 43440 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 43442 T40 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42988/_",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:42988__",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 43442 T40 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 43442 T40 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 43444 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 43547 T61 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 43547 T79 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 43547 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 43547 T48 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 44439 T80 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 44440 T80 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1391834063666/collection1
[junit4] 2> 44440 T80 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 44441 T80 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 44441 T80 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 44444 T80 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 44444 T80 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1391834063666/collection1/'
[junit4] 2> 44479 T80 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 44506 T80 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 44508 T80 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 44522 T80 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 44778 T80 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 44780 T80 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 44782 T80 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 44782 T80 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 44783 T80 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 44783 T80 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1391834063666/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty2/
[junit4] 2> 44784 T80 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 44784 T80 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty2
[junit4] 2> 44784 T80 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty2/index/
[junit4] 2> 44785 T80 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 44785 T80 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty2/index
[junit4] 2> 44786 T80 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@18354e0 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 44787 T80 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 44788 T80 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 44789 T80 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 44789 T80 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 44789 T80 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 44789 T80 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 44789 T80 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 44790 T80 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 44790 T80 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 44792 T80 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 44794 T80 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 44795 T80 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 44795 T80 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@18354e0 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 44796 T80 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 44796 T80 oass.SolrIndexSearcher.<init> Opening Searcher@14d6ef3[collection1] main
[junit4] 2> 44797 T81 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14d6ef3[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 44797 T80 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 44798 T9 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 44798 T9 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 44798 T84 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42988/_ collection:collection1 shard:shard2
[junit4] 2> 44800 T84 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 44807 T84 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 44809 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 44810 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 44810 T84 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 44810 T84 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 44810 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@172b4a6 url=http://127.0.0.1:42988/_/collection1 node=127.0.0.1:42988__ C2_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:42988/_, core=collection1, node_name=127.0.0.1:42988__}
[junit4] 2> 44811 T84 C2 P42988 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42988/_/collection1/
[junit4] 2> 44812 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 44812 T84 C2 P42988 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 44812 T84 C2 P42988 oasc.SyncStrategy.syncToMe http://127.0.0.1:42988/_/collection1/ has no replicas
[junit4] 2> 44813 T84 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42988/_/collection1/ shard2
[junit4] 2> 44813 T84 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 44816 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 44821 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 44831 T9 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 44834 T9 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 44836 T9 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47931
[junit4] 2> 44837 T9 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 44837 T9 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 44837 T9 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1391834066147
[junit4] 2> 44838 T9 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1391834066147/'
[junit4] 2> 44838 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 44854 T9 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1391834066147/solr.xml
[junit4] 2> 44886 T9 oasc.CoreContainer.<init> New CoreContainer 12258385
[junit4] 2> 44886 T9 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1391834066147/]
[junit4] 2> 44887 T9 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 44888 T9 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 44888 T9 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 44888 T9 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 44888 T9 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 44889 T9 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 44889 T9 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 44889 T9 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 44890 T9 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 44893 T9 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 44893 T9 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 44894 T9 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 44894 T9 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37513/solr
[junit4] 2> 44895 T9 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 44895 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 44897 T95 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25be1e name:ZooKeeperConnection Watcher:127.0.0.1:37513 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 44898 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 44900 T9 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 44902 T97 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15bc2a6 name:ZooKeeperConnection Watcher:127.0.0.1:37513/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 44902 T9 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 44966 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 44966 T48 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 44966 T61 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 44966 T79 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 44968 T9 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 44973 T84 oasc.ZkController.register We are http://127.0.0.1:42988/_/collection1/ and leader is http://127.0.0.1:42988/_/collection1/
[junit4] 2> 44973 T84 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42988/_
[junit4] 2> 44974 T84 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 44974 T84 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 44974 T84 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 44975 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 44976 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 44976 T84 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 44976 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 44977 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 44978 T40 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:42988/_",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:42988__",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 44981 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 45083 T97 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 45084 T48 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 45083 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 45083 T61 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 45083 T79 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 45974 T9 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47931__
[junit4] 2> 45976 T9 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47931__
[junit4] 2> 45981 T48 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 45983 T39 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 45984 T61 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 45984 T79 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 45984 T97 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 45997 T98 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 45997 T98 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 45999 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 46000 T98 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 46000 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 46001 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 46004 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 46006 T40 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:47931/_",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47931__",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 46006 T40 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 46007 T40 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 46012 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 46117 T79 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 46117 T39 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 46117 T48 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 46117 T61 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 46117 T97 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 47001 T98 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 47001 T98 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1391834066147/collection1
[junit4] 2> 47001 T98 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 47002 T98 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 47002 T98 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 47004 T98 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 47004 T98 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1391834066147/collection1/'
[junit4] 2> 47033 T98 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 47057 T98 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 47058 T98 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 47070 T98 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 47305 T98 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 47307 T98 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 47309 T98 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 47309 T98 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 47310 T98 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 47311 T98 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1391834066147/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3/
[junit4] 2> 47311 T98 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 47311 T98 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3
[junit4] 2> 47312 T98 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3/index/
[junit4] 2> 47312 T98 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 47312 T98 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3/index
[junit4] 2> 47314 T98 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@88160c lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3/index),segFN=segments_1,generation=1}
[junit4] 2> 47314 T98 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 47315 T98 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 47315 T98 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 47315 T98 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 47316 T98 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 47316 T98 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 47316 T98 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 47316 T98 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 47317 T98 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 47319 T98 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 47321 T98 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 47322 T98 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 47322 T98 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@88160c lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3/index),segFN=segments_1,generation=1}
[junit4] 2> 47323 T98 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 47323 T98 oass.SolrIndexSearcher.<init> Opening Searcher@11ce791[collection1] main
[junit4] 2> 47324 T99 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11ce791[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 47324 T98 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 47325 T9 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 47325 T9 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 47325 T102 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47931/_ collection:collection1 shard:shard1
[junit4] 2> 47329 T102 oasc.ZkController.register We are http://127.0.0.1:47931/_/collection1/ and leader is http://127.0.0.1:48684/_/collection1/
[junit4] 2> 47329 T102 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47931/_
[junit4] 2> 47330 T102 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 47330 T102 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@1a3d867 url=http://127.0.0.1:47931/_/collection1 node=127.0.0.1:47931__ C3_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:47931/_, core=collection1, node_name=127.0.0.1:47931__}
[junit4] 2> 47333 T103 C3 P47931 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 47334 T102 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 47334 T103 C3 P47931 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 47335 T103 C3 P47931 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 47335 T103 C3 P47931 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 47336 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 47337 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 47337 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 47338 T40 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 47339 T40 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "base_url":"http://127.0.0.1:47931/_",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47931__",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node3"}
[junit4] 2> 47341 T39 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 47365 T9 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 47366 T9 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 47369 T9 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34192
[junit4] 2> 47369 T9 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 47370 T9 o
[...truncated too long message...]
tServer.send4LetterWord connecting to 127.0.0.1:37513 37513
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 57374 T9 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 58221 T39 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 58313 T61 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 58313 T61 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 58399 T79 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 59321 T97 C11 P47931 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 59324 T97 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 59324 T97 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1a3d867
[junit4] 2> 59325 T97 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=6,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=2,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=5}
[junit4] 2> 59325 T97 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 59325 T97 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 59325 T97 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 59327 T97 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 59327 T97 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 59327 T97 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3;done=false>>]
[junit4] 2> 59327 T97 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3
[junit4] 2> 59328 T97 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3/index;done=false>>]
[junit4] 2> 59328 T97 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty3/index
[junit4] 2> 59328 T97 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 59328 T97 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 59329 T97 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 59329 T97 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 59402 T116 C10 P34192 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 59403 T116 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 59403 T116 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@944720
[junit4] 2> 59404 T116 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=6,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=2,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=5}
[junit4] 2> 59405 T116 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 59405 T116 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 59405 T116 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 59406 T116 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 59407 T116 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 59407 T116 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty4;done=false>>]
[junit4] 2> 59407 T116 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty4
[junit4] 2> 59408 T116 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty4/index;done=false>>]
[junit4] 2> 59408 T116 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1391834057691/jetty4/index
[junit4] 2> 59408 T116 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 59409 T116 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Lucene45, sim=DefaultSimilarity, locale=ca_ES, timezone=Europe/Amsterdam
[junit4] 2> NOTE: Linux 3.8.0-35-generic i386/Oracle Corporation 1.7.0_60-ea (32-bit)/cpus=8,threads=1,free=95734488,total=192675840
[junit4] 2> NOTE: All tests run in this JVM: [CloudSolrServerTest]
[junit4] Completed on J0 in 60.48s, 2 tests, 1 error <<< FAILURES!
[...truncated 112 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 47 suites, 279 tests, 1 error
Total time: 46 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b04 -client -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.6.0_45) - Build # 9296
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9296/
Java: 32bit/jdk1.6.0_45 -server -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
at __randomizedtesting.SeedInfo.seed([C5556E5019163D35:C15DE1A30BB3D214]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:289)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:153)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:584)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 9497 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> 191871 T1213 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-OverseerTest-1391839906497
[junit4] 2> 191871 T1213 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 191874 T1213 oas.SolrTestCaseJ4.setUp ###Starting testDoubleAssignment
[junit4] 2> 191874 T1213 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 191875 T1214 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 191975 T1213 oasc.ZkTestServer.run start zk server on port:53518
[junit4] 2> 191976 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 191979 T1220 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1697882 name:ZooKeeperConnection Watcher:127.0.0.1:53518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 191979 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 191980 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 191981 T1222 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@306c48 name:ZooKeeperConnection Watcher:127.0.0.1:53518 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 191981 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 191984 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 191986 T1224 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c9ebbb name:ZooKeeperConnection Watcher:127.0.0.1:53518 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 191986 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 191986 T1213 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 191990 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 191993 T1213 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 191996 T1213 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 191999 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 192003 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 192005 T1226 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@aa00e4 name:ZooKeeperConnection Watcher:127.0.0.1:53518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 192005 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 192007 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 192013 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 192020 T1220 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 192020 T1226 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 192021 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 192023 T1228 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a4d4e9 name:ZooKeeperConnection Watcher:127.0.0.1:53518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 192023 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 192024 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 192028 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 192036 T1213 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:53518_solr
[junit4] 2> 192036 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 192040 T1213 oasc.Overseer.start Overseer (id=91215620112580612-127.0.0.1:53518_solr-n_0000000000) starting
[junit4] 2> 192043 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 192048 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 192054 T1213 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 192058 T1230 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 192060 T1230 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 192061 T1229 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 192065 T1229 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 192066 T1229 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 192066 T1229 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 192066 T1229 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 192070 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192071 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 192071 T1220 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 192563 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 192582 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 192590 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192591 T1213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 192594 T1229 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 192599 T1220 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 192600 T1226 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 192601 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192602 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 192607 T1232 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ffcbe name:ZooKeeperConnection Watcher:127.0.0.1:53518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 192607 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 192609 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 192613 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 192618 T1220 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 192618 T1232 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 192625 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192625 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 192628 T1229 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 192628 T1229 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 192634 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192639 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 192647 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192657 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192762 T1220 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 192763 T1232 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 192767 T1213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 192775 T1220 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 192775 T1232 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 192776 T1215 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144101f8ac70005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 192783 T1213 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53518 53518
[junit4] 2> 192977 T1213 oas.SolrTestCaseJ4.tearDown ###Ending testDoubleAssignment
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 192985 T1213 oas.SolrTestCaseJ4.setUp ###Starting testPlaceholders
[junit4] 2> 192986 T1213 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 192986 T1233 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 193087 T1213 oasc.ZkTestServer.run start zk server on port:56645
[junit4] 2> 193088 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193090 T1239 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a439c5 name:ZooKeeperConnection Watcher:127.0.0.1:56645/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193090 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193091 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193093 T1241 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12f5fb6 name:ZooKeeperConnection Watcher:127.0.0.1:56645 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193093 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193125 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193127 T1243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@265039 name:ZooKeeperConnection Watcher:127.0.0.1:56645 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193127 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193127 T1213 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 193132 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 193135 T1213 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 193140 T1213 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 193144 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 193150 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193151 T1245 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30e58e name:ZooKeeperConnection Watcher:127.0.0.1:56645/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193151 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193153 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 193158 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 193162 T1239 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 193163 T1245 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 193164 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193165 T1247 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d6d6d4 name:ZooKeeperConnection Watcher:127.0.0.1:56645/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193165 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193166 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 193170 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 193177 T1213 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56645_solr
[junit4] 2> 193178 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 193187 T1213 oasc.Overseer.start Overseer (id=91215620185653252-127.0.0.1:56645_solr-n_0000000000) starting
[junit4] 2> 193190 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 193195 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 193201 T1213 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 193207 T1249 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 193209 T1249 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 193210 T1248 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 193216 T1248 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 193217 T1248 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=12 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"12",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 193217 T1248 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12]
[junit4] 2> 193218 T1248 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 193221 T1247 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 193223 T1239 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 193223 T1245 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 193712 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 193728 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 193737 T1247 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 193738 T1213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 193744 T1239 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 193745 T1245 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 193747 T1234 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144101f8f220000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 193748 T1213 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56645 56645
[junit4] 2> 193977 T1213 oas.SolrTestCaseJ4.tearDown ###Ending testPlaceholders
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 193984 T1213 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignment
[junit4] 2> 193984 T1213 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 193985 T1250 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 194085 T1213 oasc.ZkTestServer.run start zk server on port:36789
[junit4] 2> 194086 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194088 T1256 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@386ac2 name:ZooKeeperConnection Watcher:127.0.0.1:36789 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194089 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194123 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194125 T1258 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2c22 name:ZooKeeperConnection Watcher:127.0.0.1:36789 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194125 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194125 T1213 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 194137 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194138 T1260 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@af9150 name:ZooKeeperConnection Watcher:127.0.0.1:36789/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194138 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194139 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 194144 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194145 T1262 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63c7dd name:ZooKeeperConnection Watcher:127.0.0.1:36789/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194146 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194147 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 194150 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 194157 T1213 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:36789_solr
[junit4] 2> 194158 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 194162 T1213 oasc.Overseer.start Overseer (id=91215620251123715-127.0.0.1:36789_solr-n_0000000000) starting
[junit4] 2> 194167 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 194173 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 194179 T1213 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 194182 T1264 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 194183 T1213 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 194185 T1264 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 194186 T1263 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 194190 T1213 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 194193 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 194198 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194200 T1266 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b15f85 name:ZooKeeperConnection Watcher:127.0.0.1:36789/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194200 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194202 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 194206 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 194211 T1266 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 194212 T1260 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 194214 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194217 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 194218 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 194218 T1263 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 194218 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 194222 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194223 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 194224 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 194715 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 194730 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 194739 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194743 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194744 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 194749 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 194750 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 194750 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 194753 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194859 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 194859 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 195243 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 195256 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 195267 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195267 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195267 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195270 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 195273 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195277 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 195277 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 195278 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 195282 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195385 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 195385 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 195773 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 195786 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 195796 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195797 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195797 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195801 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 195802 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195813 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 195813 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 195813 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 195816 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195920 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 195920 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 196301 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 196313 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196314 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196314 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196316 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 196318 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 196318 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 196318 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 196323 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196428 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 196428 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 196814 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 196832 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196832 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196832 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196834 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 196835 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 196836 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 196836 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 196839 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196942 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 196943 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 197332 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 197343 T1266 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 197346 T1213 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:36789 36789
[junit4] 2> 197451 T1213 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 197458 T1213 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 197459 T1213 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 197460 T1267 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 197560 T1213 oasc.ZkTestServer.run start zk server on port:56311
[junit4] 2> 197561 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197683 T1273 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bc4c37 name:ZooKeeperConnection Watcher:127.0.0.1:56311 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197684 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197724 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197726 T1275 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@196d484 name:ZooKeeperConnection Watcher:127.0.0.1:56311 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197726 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197727 T1213 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 197730 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197732 T1277 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d003b name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197732 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197733 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 197735 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197737 T1279 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a21149 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197737 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197738 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 197741 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 197752 T1213 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56311_solr
[junit4] 2> 197752 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 197756 T1213 oasc.Overseer.start Overseer (id=91215620478926851-127.0.0.1:56311_solr-n_0000000000) starting
[junit4] 2> 197759 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 197765 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 197770 T1213 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 197773 T1281 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 197774 T1213 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 197775 T1281 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 197776 T1280 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 197780 T1213 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 197784 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197788 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197790 T1283 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@874c04 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197790 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197792 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197795 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
[junit4] 2> 197800 T1283 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 197800 T1277 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 197801 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197802 T1285 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@103a2d8 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197803 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197804 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197808 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 197819 T1285 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 197820 T1277 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 197820 T1283 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 197820 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197822 T1287 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d12717 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197822 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197824 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197828 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 197833 T1285 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 197833 T1277 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 197834 T1283 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 197834 T1287 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 197835 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197836 T1289 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a799c1 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197837 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197838 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197842 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
[junit4] 2> 197849 T1287 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197849 T1283 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197850 T1289 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197851 T1277 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197851 T1285 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197851 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197852 T1291 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67ebee name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197853 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197854 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197858 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
[junit4] 2> 197862 T1285 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197864 T1287 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197864 T1289 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197865 T1283 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197865 T1291 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197866 T1277 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197867 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197868 T1293 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8b05ad name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197868 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197870 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197876 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
[junit4] 2> 197883 T1287 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197883 T1283 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197883 T1289 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197884 T1285 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197884 T1293 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197884 T1291 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197885 T1277 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197885 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197893 T1295 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e04525 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197893 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197895 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197899 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
[junit4] 2> 197905 T1287 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197906 T1291 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197907 T1283 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197910 T1289 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197910 T1293 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197910 T1277 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197910 T1285 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197911 T1295 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197912 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197913 T1297 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c53ff name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197913 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197915 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197918 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
[junit4] 2> 197925 T1277 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197925 T1291 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197925 T1297 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197926 T1283 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197926 T1289 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197925 T1295 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197926 T1293 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197927 T1287 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197927 T1285 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197928 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197929 T1299 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@db33bc name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197929 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197931 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197934 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
[junit4] 2> 197942 T1285 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197943 T1287 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197943 T1299 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197944 T1293 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197944 T1295 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197944 T1283 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197943 T1291 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197944 T1289 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197945 T1277 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197946 T1297 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197946 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197948 T1301 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@432225 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197948 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197950 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197953 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
[junit4] 2> 197961 T1287 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197961 T1295 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197961 T1283 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1289 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1277 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1293 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1297 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1299 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197963 T1301 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197963 T1291 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197964 T1285 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197964 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197965 T1303 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@179ad95 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197966 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197967 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197971 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
[junit4] 2> 197980 T1287 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197980 T1293 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1285 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1299 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1295 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1283 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1289 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197982 T1277 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1301 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197982 T1297 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197983 T1291 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197983 T1303 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197984 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197985 T1305 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dc68a2 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197985 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197987 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197994 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
[junit4] 2> 198003 T1287 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198004 T1291 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198004 T1289 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198004 T1283 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1277 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1301 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1285 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1299 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1295 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1303 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198006 T1305 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198007 T1293 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198008 T1297 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198008 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198009 T1307 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7caa0f name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198009 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198011 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198014 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
[junit4] 2> 198024 T1287 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198025 T1303 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198026 T1299 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198028 T1297 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198028 T1285 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198029 T1291 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198029 T1289 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198029 T1277 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198029 T1305 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198030 T1283 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198030 T1301 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198030 T1307 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198031 T1295 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198031 T1293 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198032 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198033 T1309 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d89d8c name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198033 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198036 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198039 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
[junit4] 2> 198048 T1307 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198049 T1287 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198050 T1297 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198050 T1305 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1295 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1301 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198050 T1283 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1299 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1285 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1289 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1303 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1291 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198052 T1293 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198052 T1277 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198053 T1309 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198054 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198055 T1311 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15a39a4 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198056 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198057 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198061 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
[junit4] 2> 198069 T1291 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198071 T1309 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198071 T1301 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198071 T1295 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198072 T1297 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198072 T1277 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198072 T1293 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198072 T1305 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1289 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1285 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1283 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1311 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1307 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198074 T1303 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1299 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198082 T1287 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198083 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198084 T1313 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19de1ba name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198084 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198086 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198089 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
[junit4] 2> 198113 T1295 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198114 T1287 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1283 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1297 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1277 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1293 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198121 T1301 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1291 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198122 T1299 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198122 T1303 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198121 T1289 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198122 T1313 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198122 T1285 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198128 T1305 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198128 T1307 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198129 T1309 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198129 T1311 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198129 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198131 T1315 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47ab39 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198131 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198132 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198135 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
[junit4] 2> 198146 T1287 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198146 T1277 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198146 T1295 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198146 T1303 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198146 T1293 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198147 T1311 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1301 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1299 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1285 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1307 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1309 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198149 T1297 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198149 T1289 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198149 T1305 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198150 T1313 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198150 T1315 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198151 T1291 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198151 T1283 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198151 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198152 T1317 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b860b7 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198153 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198154 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198158 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
[junit4] 2> 198165 T1291 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198172 T1287 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1285 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1277 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1305 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1301 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198174 T1293 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1283 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198174 T1317 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198174 T1299 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1303 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1307 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1313 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1309 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1295 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198176 T1315 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198176 T1289 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198177 T1311 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198177 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198177 T1297 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198178 T1319 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d55c36 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198179 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198180 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198183 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
[junit4] 2> 198193 T1305 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198195 T1303 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198196 T1287 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198196 T1277 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198197 T1307 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1319 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1299 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1301 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1315 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1309 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1317 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1289 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1285 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1283 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198200 T1293 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1297 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1313 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1295 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198201 T1311 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198202 T1291 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198202 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198204 T1321 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19a01f9 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198204 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198206 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198211 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
[junit4] 2> 198220 T1287 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198220 T1295 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198221 T1317 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198223 T1291 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198224 T1301 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198225 T1283 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198225 T1277 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198226 T1303 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1305 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1307 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1289 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1285 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1297 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198228 T1299 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198228 T1309 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198228 T1313 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198229 T1293 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198230 T1321 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198230 T1315 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198230 T1311 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198231 T1319 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198232 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198233 T1323 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dd36f3 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198233 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198235 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198238 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
[junit4] 2> 198249 T1319 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198250 T1301 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198251 T1313 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198252 T1287 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198252 T1291 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198253 T1285 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198253 T1277 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198253 T1289 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1305 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1317 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1283 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1315 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1297 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198255 T1307 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198255 T1309 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1303 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1299 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198255 T1323 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198255 T1321 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198257 T1295 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198257 T1293 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198256 T1311 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198263 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198264 T1325 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fbf68 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198265 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198266 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198269 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
[junit4] 2> 198282 T1287 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198283 T1277 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198284 T1289 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198284 T1309 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198284 T1285 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198284 T1291 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1319 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1293 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1317 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1299 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1305 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1283 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1301 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1323 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1313 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198287 T1295 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198287 T1321 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198287 T1307 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1303 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1315 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198287 T1325 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198289 T1311 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198289 T1297 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198290 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198291 T1327 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8aa935 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198291 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198293 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198298 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
[junit4] 2> 198311 T1319 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198311 T1315 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198319 T1287 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198319 T1277 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1323 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1285 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1301 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1317 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1291 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198321 T1293 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1299 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1309 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1303 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1283 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1305 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198323 T1307 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1289 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198323 T1295 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198323 T1321 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198324 T1313 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198323 T1325 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198325 T1327 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198326 T1311 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198326 T1297 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198326 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198328 T1329 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13f333c name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198328 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198329 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198336 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
[junit4] 2> 198351 T1287 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198352 T1307 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198352 T1303 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198353 T1311 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198353 T1317 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198353 T1309 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1277 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1289 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1295 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1321 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1297 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198355 T1315 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198355 T1325 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198355 T1291 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198355 T1293 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198356 T1319 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198356 T1327 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198356 T1323 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198356 T1283 oascc.ZkStateReader$3.process Updating live nodes... (24)
[...truncated too long message...]
:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 230246 T1771 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 230246 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 230250 T1772 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 230273 T1765 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 230275 T1765 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 230277 T1759 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 230281 T1213 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48107 48107
[junit4] 2> 230437 T1213 oas.SolrTestCaseJ4.tearDown ###Ending testOverseerFailure
[junit4] 2> 230438 T1772 oasc.SolrException.log ERROR Cleaning up collection in zk was interrupted:collection1:java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:119)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:87)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:171)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:655)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.removeCore(Overseer.java:875)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:226)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 230438 T1772 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue-work/qn-
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:240)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:237)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:237)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:284)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:271)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:188)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerFailure -Dtests.seed=C5556E5019163D35 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_NZ -Dtests.timezone=Asia/Damascus -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 3.21s J0 | OverseerTest.testOverseerFailure <<<
[junit4] > Throwable #1: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] > at __randomizedtesting.SeedInfo.seed([C5556E5019163D35:C15DE1A30BB3D214]:0)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:289)
[junit4] > at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:153)
[junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:584)
[junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4] 2> 230457 T1213 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 230457 T1213 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 231776 T1765 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 233458 T1213 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Lucene46: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=en_NZ, timezone=Asia/Damascus
[junit4] 2> NOTE: Linux 3.8.0-35-generic i386/Sun Microsystems Inc. 1.6.0_45 (32-bit)/cpus=8,threads=1,free=33230816,total=99135488
[junit4] 2> NOTE: All tests run in this JVM: [FieldMutatingUpdateProcessorTest, DocumentBuilderTest, TriLevelCompositeIdRoutingTest, TestPostingsSolrHighlighter, TestSolrDeletionPolicy1, ZkCLITest, DirectUpdateHandlerOptimizeTest, ShardRoutingTest, LegacyHTMLStripCharFilterTest, SampleTest, RegexBoostProcessorTest, BlockCacheTest, QueryEqualityTest, TestCoreContainer, ScriptEngineTest, MBeansHandlerTest, OverseerRolesTest, OverseerTest]
[junit4] Completed on J0 in 41.61s, 8 tests, 1 error <<< FAILURES!
[...truncated 850 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 358 suites, 1554 tests, 1 error, 36 ignored (8 assumptions)
Total time: 59 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.6.0_45 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure