You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/06/11 09:20:13 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_05) - Build #
10525 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10525/
Java: 32bit/jdk1.8.0_05 -client -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.cloud.CustomCollectionTest.testDistribSearch
Error Message:
createshard the collection time out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: createshard the collection time out:180s
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI(CustomCollectionTest.java:274)
at org.apache.solr.cloud.CustomCollectionTest.doTest(CustomCollectionTest.java:127)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11300 lines...]
[junit4] Suite: org.apache.solr.cloud.CustomCollectionTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/init-core-data-001
[junit4] 2> 909289 T3838 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 909290 T3838 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 909293 T3838 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 909294 T3838 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 909294 T3839 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 909394 T3838 oasc.ZkTestServer.run start zk server on port:60713
[junit4] 2> 909395 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 909599 T3845 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17d36c0 name:ZooKeeperConnection Watcher:127.0.0.1:60713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 909599 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 909600 T3838 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 909603 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 909604 T3847 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a5382 name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 909605 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 909605 T3838 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 909607 T3838 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 909608 T3838 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 909609 T3838 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 909611 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 909611 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 909620 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 909621 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 909624 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 909625 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 909626 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 909627 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 909628 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 909629 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 909630 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 909631 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 909632 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 909633 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 909639 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 909639 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 909641 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 909642 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 909644 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 909644 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 909646 T3838 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 909646 T3838 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 909649 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 909650 T3849 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51c711 name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 909650 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 909928 T3838 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 909931 T3838 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 909934 T3838 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:52119
[junit4] 2> 909935 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 909935 T3838 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 909936 T3838 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002
[junit4] 2> 909936 T3838 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002/'
[junit4] 2> 909951 T3838 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002/solr.xml
[junit4] 2> 909966 T3838 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002
[junit4] 2> 909966 T3838 oasc.CoreContainer.<init> New CoreContainer 11558613
[junit4] 2> 909967 T3838 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002/]
[junit4] 2> 909967 T3838 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 909968 T3838 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 909968 T3838 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 909969 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 909969 T3838 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 909969 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 909970 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 909970 T3838 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 909970 T3838 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 909972 T3838 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 909973 T3838 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 909973 T3838 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 909973 T3838 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60713/solr
[junit4] 2> 909974 T3838 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 909974 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 909976 T3860 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16074cb name:ZooKeeperConnection Watcher:127.0.0.1:60713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 909977 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 909979 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 909980 T3862 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ace466 name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 909980 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 909981 T3838 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 909984 T3838 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 909986 T3838 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 909988 T3838 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 909990 T3838 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 909992 T3838 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 910000 T3838 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52119_
[junit4] 2> 910001 T3838 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52119_
[junit4] 2> 910003 T3838 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 910005 T3838 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 910007 T3838 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52119_
[junit4] 2> 910007 T3838 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 910008 T3838 oasc.Overseer.start Overseer (id=91912264040710148-127.0.0.1:52119_-n_0000000000) starting
[junit4] 2> 910010 T3838 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 910016 T3864 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 910016 T3838 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 910018 T3838 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 910019 T3838 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 910020 T3863 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 910021 T3838 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002
[junit4] 2> 910026 T3838 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002/conf/
[junit4] 2> 910131 T3838 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 910133 T3838 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 910133 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 910137 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 910140 T3867 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cc7367 name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 910140 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 910141 T3838 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 910144 T3838 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 910426 T3838 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 910427 T3838 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 910430 T3838 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 910432 T3838 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:52023
[junit4] 2> 910433 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 910434 T3838 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 910434 T3838 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-003
[junit4] 2> 910435 T3838 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-003/'
[junit4] 2> 910457 T3838 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-003/solr.xml
[junit4] 2> 910468 T3838 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-003
[junit4] 2> 910469 T3838 oasc.CoreContainer.<init> New CoreContainer 9812898
[junit4] 2> 910470 T3838 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-003/]
[junit4] 2> 910471 T3838 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 910471 T3838 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 910472 T3838 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 910473 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 910473 T3838 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 910474 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 910474 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 910474 T3838 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 910475 T3838 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 910476 T3838 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 910477 T3838 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 910477 T3838 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 910478 T3838 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60713/solr
[junit4] 2> 910478 T3838 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 910479 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 910481 T3878 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fe6c98 name:ZooKeeperConnection Watcher:127.0.0.1:60713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 910481 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 910484 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 910485 T3880 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32ed69 name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 910485 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 910491 T3838 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 911494 T3838 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52023_
[junit4] 2> 911498 T3838 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52023_
[junit4] 2> 911510 T3838 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-003
[junit4] 2> 911524 T3838 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-003/conf/
[junit4] 2> 911641 T3838 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 911643 T3838 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 911643 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 911897 T3838 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 911897 T3838 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 911900 T3838 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 911902 T3838 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:53574
[junit4] 2> 911904 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 911905 T3838 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 911905 T3838 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004
[junit4] 2> 911906 T3838 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004/'
[junit4] 2> 911925 T3838 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004/solr.xml
[junit4] 2> 911934 T3838 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004
[junit4] 2> 911935 T3838 oasc.CoreContainer.<init> New CoreContainer 24821116
[junit4] 2> 911935 T3838 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004/]
[junit4] 2> 911935 T3838 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 911936 T3838 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 911936 T3838 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 911936 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 911937 T3838 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 911937 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 911937 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 911938 T3838 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 911938 T3838 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 911940 T3838 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 911940 T3838 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 911940 T3838 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 911941 T3838 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60713/solr
[junit4] 2> 911941 T3838 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 911942 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 911945 T3892 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e22059 name:ZooKeeperConnection Watcher:127.0.0.1:60713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 911946 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 911952 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 911953 T3894 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37ffbe name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 911954 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 911959 T3838 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 912961 T3838 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53574_
[junit4] 2> 912963 T3838 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53574_
[junit4] 2> 912969 T3838 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004
[junit4] 2> 912976 T3838 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004/conf/
[junit4] 2> 913113 T3838 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 913114 T3838 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 913115 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 913381 T3838 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 913381 T3838 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 913385 T3838 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 913386 T3838 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:36503
[junit4] 2> 913388 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 913388 T3838 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 913388 T3838 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-005
[junit4] 2> 913389 T3838 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-005/'
[junit4] 2> 913405 T3838 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-005/solr.xml
[junit4] 2> 913415 T3838 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-005
[junit4] 2> 913416 T3838 oasc.CoreContainer.<init> New CoreContainer 32311455
[junit4] 2> 913416 T3838 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-005/]
[junit4] 2> 913417 T3838 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 913417 T3838 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 913417 T3838 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 913417 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 913418 T3838 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 913418 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 913418 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 913435 T3838 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 913435 T3838 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 913437 T3838 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 913438 T3838 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 913438 T3838 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 913438 T3838 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60713/solr
[junit4] 2> 913439 T3838 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 913439 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 913441 T3906 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1607848 name:ZooKeeperConnection Watcher:127.0.0.1:60713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 913442 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 913445 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 913450 T3908 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10e46 name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 913450 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 913455 T3838 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 914457 T3838 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36503_
[junit4] 2> 914458 T3838 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36503_
[junit4] 2> 914464 T3838 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-005
[junit4] 2> 914474 T3838 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-005/conf/
[junit4] 2> 914586 T3838 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 914587 T3838 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 914588 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 914879 T3838 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 914880 T3838 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 914883 T3838 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 914884 T3838 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50021
[junit4] 2> 914885 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 914886 T3838 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 914886 T3838 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006
[junit4] 2> 914886 T3838 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/'
[junit4] 2> 914900 T3838 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/solr.xml
[junit4] 2> 914909 T3838 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006
[junit4] 2> 914910 T3838 oasc.CoreContainer.<init> New CoreContainer 19026435
[junit4] 2> 914910 T3838 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/]
[junit4] 2> 914914 T3838 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 914915 T3838 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 914915 T3838 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 914916 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 914916 T3838 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 914916 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 914916 T3838 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 914917 T3838 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 914917 T3838 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 914919 T3838 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 914920 T3838 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 914920 T3838 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 914921 T3838 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60713/solr
[junit4] 2> 914921 T3838 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 914922 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 914924 T3920 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19e23fa name:ZooKeeperConnection Watcher:127.0.0.1:60713 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 914924 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 914926 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 914928 T3922 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@96e4fb name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 914928 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 914957 T3838 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 915961 T3838 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50021_
[junit4] 2> 915963 T3838 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50021_
[junit4] 2> 915974 T3838 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006
[junit4] 2> 915989 T3838 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/conf/
[junit4] 2> 916142 T3838 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 916143 T3838 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 916144 T3838 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 916149 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 916151 T3925 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ab6dea name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 916151 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 916153 T3838 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 916158 T3838 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 916160 T3927 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b421c0 name:ZooKeeperConnection Watcher:127.0.0.1:60713/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 916160 T3838 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 916162 T3838 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 916289 T3874 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=4&shards=a,b,c&maxShardsPerNode=4&router.name=implicit&name=implicitcoll0&action=CREATE&wt=javabin&version=2
[junit4] 2> 916290 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916291 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916291 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916291 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916291 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916291 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916292 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916292 T3864 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"implicitcoll0",
[junit4] 2> "replicationFactor":"4",
[junit4] 2> "maxShardsPerNode":"4",
[junit4] 2> "shards":"a,b,c",
[junit4] 2> "router.name":"implicit"}
[junit4] 2> 916293 T3928 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"implicitcoll0",
[junit4] 2> "replicationFactor":"4",
[junit4] 2> "maxShardsPerNode":"4",
[junit4] 2> "shards":"a,b,c",
[junit4] 2> "router.name":"implicit"}
[junit4] 2> 916295 T3928 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
[junit4] 2> 916295 T3928 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/implicitcoll0
[junit4] 2> 916295 T3928 oascc.SolrZkClient.makePath makePath: /collections/implicitcoll0
[junit4] 2> 916300 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916302 T3863 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 916302 T3863 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: implicitcoll0
[junit4] 2> 916303 T3863 oasc.Overseer$ClusterStateUpdater.createCollection Create collection implicitcoll0 with shards [a, b, c]
[junit4] 2> 916305 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916306 T3908 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916306 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916306 T3925 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916306 T3927 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916306 T3922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916306 T3880 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916306 T3894 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916306 T3867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916400 T3928 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection implicitcoll0, shardNames [a, b, c] , replicationFactor : 4
[junit4] 2> 916401 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_a_replica1 as part of slice a of collection implicitcoll0 on 127.0.0.1:52119_
[junit4] 2> 916402 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_a_replica2 as part of slice a of collection implicitcoll0 on 127.0.0.1:53574_
[junit4] 2> 916403 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_a_replica3 as part of slice a of collection implicitcoll0 on 127.0.0.1:50021_
[junit4] 2> 916405 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_a_replica4 as part of slice a of collection implicitcoll0 on 127.0.0.1:36503_
[junit4] 2> 916407 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_b_replica1 as part of slice b of collection implicitcoll0 on 127.0.0.1:52023_
[junit4] 2> 916410 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_b_replica2 as part of slice b of collection implicitcoll0 on 127.0.0.1:52119_
[junit4] 2> 916412 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_b_replica3 as part of slice b of collection implicitcoll0 on 127.0.0.1:53574_
[junit4] 2> 916419 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_b_replica4 as part of slice b of collection implicitcoll0 on 127.0.0.1:50021_
[junit4] 2> 916420 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_c_replica1 as part of slice c of collection implicitcoll0 on 127.0.0.1:36503_
[junit4] 2> 916437 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_c_replica2 as part of slice c of collection implicitcoll0 on 127.0.0.1:52023_
[junit4] 2> 916438 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_c_replica3 as part of slice c of collection implicitcoll0 on 127.0.0.1:52119_
[junit4] 2> 916448 T3928 oasc.OverseerCollectionProcessor.createCollection Creating shard implicitcoll0_c_replica4 as part of slice c of collection implicitcoll0 on 127.0.0.1:53574_
[junit4] 2> 916599 T3915 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_a_replica3&action=CREATE&numShards=3&collection=implicitcoll0&shard=a&wt=javabin&version=2
[junit4] 2> 916602 T3915 oasc.ZkController.publish publishing core=implicitcoll0_a_replica3 state=down collection=implicitcoll0
[junit4] 2> 916604 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916607 T3915 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916608 T3863 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 916608 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:50021",
[junit4] 2> "core":"implicitcoll0_a_replica3",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50021_",
[junit4] 2> "shard":"a",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916610 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916678 T3887 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_a_replica2&action=CREATE&numShards=3&collection=implicitcoll0&shard=a&wt=javabin&version=2
[junit4] 2> 916680 T3856 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_a_replica1&action=CREATE&numShards=3&collection=implicitcoll0&shard=a&wt=javabin&version=2
[junit4] 2> 916680 T3887 oasc.ZkController.publish publishing core=implicitcoll0_a_replica2 state=down collection=implicitcoll0
[junit4] 2> 916681 T3887 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916685 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916686 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:53574",
[junit4] 2> "core":"implicitcoll0_a_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:53574_",
[junit4] 2> "shard":"a",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916687 T3875 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_b_replica1&action=CREATE&numShards=3&collection=implicitcoll0&shard=b&wt=javabin&version=2
[junit4] 2> 916688 T3856 oasc.ZkController.publish publishing core=implicitcoll0_a_replica1 state=down collection=implicitcoll0
[junit4] 2> 916690 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916690 T3856 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916691 T3875 oasc.ZkController.publish publishing core=implicitcoll0_b_replica1 state=down collection=implicitcoll0
[junit4] 2> 916692 T3875 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916692 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52119",
[junit4] 2> "core":"implicitcoll0_a_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:52119_",
[junit4] 2> "shard":"a",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916694 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916694 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52023",
[junit4] 2> "core":"implicitcoll0_b_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:52023_",
[junit4] 2> "shard":"b",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916696 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916712 T3888 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_c_replica4&action=CREATE&numShards=3&collection=implicitcoll0&shard=c&wt=javabin&version=2
[junit4] 2> 916712 T3901 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_a_replica4&action=CREATE&numShards=3&collection=implicitcoll0&shard=a&wt=javabin&version=2
[junit4] 2> 916715 T3901 oasc.ZkController.publish publishing core=implicitcoll0_a_replica4 state=down collection=implicitcoll0
[junit4] 2> 916716 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916716 T3901 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916716 T3888 oasc.ZkController.publish publishing core=implicitcoll0_c_replica4 state=down collection=implicitcoll0
[junit4] 2> 916716 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:36503",
[junit4] 2> "core":"implicitcoll0_a_replica4",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:36503_",
[junit4] 2> "shard":"a",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916718 T3888 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916718 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916721 T3918 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_b_replica4&action=CREATE&numShards=3&collection=implicitcoll0&shard=b&wt=javabin&version=2
[junit4] 2> 916724 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:53574",
[junit4] 2> "core":"implicitcoll0_c_replica4",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:53574_",
[junit4] 2> "shard":"c",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916725 T3918 oasc.ZkController.publish publishing core=implicitcoll0_b_replica4 state=down collection=implicitcoll0
[junit4] 2> 916725 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916726 T3918 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916726 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:50021",
[junit4] 2> "core":"implicitcoll0_b_replica4",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50021_",
[junit4] 2> "shard":"b",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916728 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916732 T3857 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_b_replica2&action=CREATE&numShards=3&collection=implicitcoll0&shard=b&wt=javabin&version=2
[junit4] 2> 916733 T3857 oasc.ZkController.publish publishing core=implicitcoll0_b_replica2 state=down collection=implicitcoll0
[junit4] 2> 916734 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916734 T3857 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916738 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52119",
[junit4] 2> "core":"implicitcoll0_b_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:52119_",
[junit4] 2> "shard":"b",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916738 T3873 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_c_replica2&action=CREATE&numShards=3&collection=implicitcoll0&shard=c&wt=javabin&version=2
[junit4] 2> 916741 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916742 T3890 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_b_replica3&action=CREATE&numShards=3&collection=implicitcoll0&shard=b&wt=javabin&version=2
[junit4] 2> 916743 T3873 oasc.ZkController.publish publishing core=implicitcoll0_c_replica2 state=down collection=implicitcoll0
[junit4] 2> 916744 T3890 oasc.ZkController.publish publishing core=implicitcoll0_b_replica3 state=down collection=implicitcoll0
[junit4] 2> 916744 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916744 T3873 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916745 T3890 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916746 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52023",
[junit4] 2> "core":"implicitcoll0_c_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:52023_",
[junit4] 2> "shard":"c",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916749 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916749 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:53574",
[junit4] 2> "core":"implicitcoll0_b_replica3",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:53574_",
[junit4] 2> "shard":"b",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916751 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916759 T3855 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_c_replica3&action=CREATE&numShards=3&collection=implicitcoll0&shard=c&wt=javabin&version=2
[junit4] 2> 916761 T3855 oasc.ZkController.publish publishing core=implicitcoll0_c_replica3 state=down collection=implicitcoll0
[junit4] 2> 916764 T3855 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916764 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916764 T3902 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=implicitcoll0_c_replica1&action=CREATE&numShards=3&collection=implicitcoll0&shard=c&wt=javabin&version=2
[junit4] 2> 916765 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52119",
[junit4] 2> "core":"implicitcoll0_c_replica3",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:52119_",
[junit4] 2> "shard":"c",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916766 T3902 oasc.ZkController.publish publishing core=implicitcoll0_c_replica1 state=down collection=implicitcoll0
[junit4] 2> 916767 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916767 T3902 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 916768 T3863 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:36503",
[junit4] 2> "core":"implicitcoll0_c_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:36503_",
[junit4] 2> "shard":"c",
[junit4] 2> "collection":"implicitcoll0",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 916770 T3862 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 916873 T3908 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916874 T3880 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916874 T3925 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916874 T3922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916873 T3862 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916873 T3894 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916873 T3867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 916873 T3927 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 917608 T3915 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_a_replica3
[junit4] 2> 917609 T3915 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917611 T3915 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917611 T3915 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917612 T3915 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917613 T3915 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/implicitcoll0_a_replica3/'
[junit4] 2> 917648 T3915 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 917664 T3915 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 917665 T3915 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 917675 T3915 oass.IndexSchema.readSchema [implicitcoll0_a_replica3] Schema name=test
[junit4] 2> 917682 T3887 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_a_replica2
[junit4] 2> 917683 T3887 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917684 T3887 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917684 T3887 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917685 T3887 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917685 T3887 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004/implicitcoll0_a_replica2/'
[junit4] 2> 917690 T3856 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_a_replica1
[junit4] 2> 917691 T3856 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917692 T3856 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917692 T3875 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_b_replica1
[junit4] 2> 917692 T3856 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917693 T3875 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917694 T3856 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917694 T3875 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917694 T3856 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002/implicitcoll0_a_replica1/'
[junit4] 2> 917695 T3875 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917699 T3875 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917699 T3875 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-003/implicitcoll0_b_replica1/'
[junit4] 2> 917717 T3901 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_a_replica4
[junit4] 2> 917718 T3901 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917718 T3888 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_c_replica4
[junit4] 2> 917719 T3901 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917719 T3901 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917720 T3888 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917721 T3888 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917721 T3901 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917722 T3888 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917722 T3901 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-005/implicitcoll0_a_replica4/'
[junit4] 2> 917736 T3857 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_b_replica2
[junit4] 2> 917738 T3857 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917737 T3918 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_b_replica4
[junit4] 2> 917739 T3918 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917740 T3918 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917741 T3888 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917741 T3857 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917742 T3857 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917745 T3873 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_c_replica2
[junit4] 2> 917741 T3918 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917746 T3890 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_b_replica3
[junit4] 2> 917741 T3887 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 917741 T3888 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004/implicitcoll0_c_replica4/'
[junit4] 2> 917748 T3890 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917749 T3918 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917749 T3918 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/implicitcoll0_b_replica4/'
[junit4] 2> 917750 T3890 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917751 T3890 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917752 T3890 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917753 T3890 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-004/implicitcoll0_b_replica3/'
[junit4] 2> 917747 T3873 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917754 T3857 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917754 T3873 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917754 T3857 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002/implicitcoll0_b_replica2/'
[junit4] 2> 917755 T3873 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917768 T3855 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_c_replica3
[junit4] 2> 917768 T3902 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for implicitcoll0_c_replica1
[junit4] 2> 917769 T3855 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917770 T3902 oasc.ZkController.createCollectionZkNode Check for collection zkNode:implicitcoll0
[junit4] 2> 917770 T3873 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917771 T3873 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-003/implicitcoll0_c_replica2/'
[junit4] 2> 917773 T3902 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917773 T3855 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 917774 T3855 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917774 T3902 oascc.ZkStateReader.readConfigName Load collection config from:/collections/implicitcoll0
[junit4] 2> 917779 T3902 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917779 T3902 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-005/implicitcoll0_c_replica1/'
[junit4] 2> 917782 T3855 oascc.ZkStateReader.readConfigName path=/collections/implicitcoll0 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 917783 T3855 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-002/implicitcoll0_c_replica3/'
[junit4] 2> 917796 T3887 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 917798 T3875 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 917809 T3887 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 917832 T3887 oass.IndexSchema.readSchema [implicitcoll0_a_replica2] Schema name=test
[junit4] 2> 917835 T3856 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 917855 T3875 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 917856 T3875 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 917870 T3918 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 917892 T3856 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 917902 T3875 oass.IndexSchema.readSchema [implicitcoll0_b_replica1] Schema name=test
[junit4] 2> 917915 T3901 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 917917 T3856 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 917934 T3888 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 917977 T3918 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 917978 T3902 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 917987 T3856 oass.IndexSchema.readSchema [implicitcoll0_a_replica1] Schema name=test
[junit4] 2> 918004 T3918 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 918009 T3888 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 918014 T3901 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 918015 T3857 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 918026 T3855 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 918027 T3888 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 918026 T3901 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 918043 T3902 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 918038 T3918 oass.IndexSchema.readSchema [implicitcoll0_b_replica4] Schema name=test
[junit4] 2> 918054 T3902 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 918051 T3873 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 918072 T3857 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 918096 T3855 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 918105 T3888 oass.IndexSchema.readSchema [implicitcoll0_c_replica4] Schema name=test
[junit4] 2> 918107 T3857 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 918107 T3890 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 918109 T3902 oass.IndexSchema.readSchema [implicitcoll0_c_replica1] Schema name=test
[junit4] 2> 918120 T3855 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 918124 T3857 oass.IndexSchema.readSchema [implicitcoll0_b_replica2] Schema name=test
[junit4] 2> 918126 T3855 oass.IndexSchema.readSchema [implicitcoll0_c_replica3] Schema name=test
[junit4] 2> 918165 T3901 oass.IndexSchema.readSchema [implicitcoll0_a_replica4] Schema name=test
[junit4] 2> 918177 T3873 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 918193 T3873 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 918202 T3890 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 918205 T3890 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 918256 T3915 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 918259 T3873 oass.IndexSchema.readSchema [implicitcoll0_c_replica2] Schema name=test
[junit4] 2> 918283 T3915 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 918286 T3915 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 918288 T3890 oass.IndexSchema.readSchema [implicitcoll0_b_replica3] Schema name=test
[junit4] 2> 918362 T3915 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 918392 T3915 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 918403 T3915 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 918415 T3915 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 918420 T3915 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 918421 T3915 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 918427 T3915 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 918428 T3915 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 918429 T3915 oasc.CoreContainer.create Creating SolrCore 'implicitcoll0_a_replica3' using configuration from collection implicitcoll0
[junit4] 2> 918429 T3915 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 918430 T3915 oasc.SolrCore.<init> [implicitcoll0_a_replica3] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/implicitcoll0_a_replica3/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/implicitcoll0_a_replica3/data/
[junit4] 2> 918431 T3915 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@9d6dcb
[junit4] 2> 918432 T3915 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/implicitcoll0_a_replica3/data
[junit4] 2> 918433 T3915 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/implicitcoll0_a_replica3/data/index/
[junit4] 2> 918433 T3915 oasc.SolrCore.initIndex WARN [implicitcoll0_a_replica3] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/implicitcoll0_a_replica3/data/index' doesn't exist. Creating new index...
[junit4] 2> 918458 T3915 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001/tempDir-006/implicitcoll0_a_replica3/data/index
[junit4] 2> 918459 T3915 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5361433022933566]
[junit4] 2> 918502 T3915 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@18ea643 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19b4597),segFN=segments_1,generation=1}
[junit4] 2> 918502 T3915 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 918512 T3915 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 918513 T3915 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 918514 T3915 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 918514 T3915 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 918515 T3915 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 918515 T3915 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 918516 T3915 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 918517 T3915 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 918517 T3915 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 918518 T3915 oasc.RequestHandlers.initHandlersFromConfig created standard: s
[...truncated too long message...]
3)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1175063 T3908 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/implicitcoll3/leader_elect/a/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1175064 T3922 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/implicitcoll4/leader_elect/b/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1175065 T3908 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1175066 T3908 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/implicitcoll2/leader_elect/b/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CustomCollectionTest-5528526877DCAEAF-001
[junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=uk, timezone=Asia/Irkutsk
[junit4] 2> NOTE: Linux 3.8.0-41-generic i386/Oracle Corporation 1.8.0_05 (32-bit)/cpus=8,threads=1,free=133437232,total=281395200
[junit4] 2> NOTE: All tests run in this JVM: [DocValuesMultiTest, AnalysisErrorHandlingTest, TestIndexingPerformance, SolrIndexConfigTest, HardAutoCommitTest, DeleteReplicaTest, TestSolr4Spatial, DateMathParserTest, TestManagedSchemaFieldResource, TestDynamicFieldCollectionResource, TestDistributedSearch, PrimUtilsTest, DOMUtilTest, CurrencyFieldOpenExchangeTest, TestRestManager, TestRangeQuery, TestManagedSchema, ModifyConfFileTest, PathHierarchyTokenizerFactoryTest, AsyncMigrateRouteKeyTest, TestDocumentBuilder, TestQueryTypes, DistributedSuggestComponentTest, TestCodecSupport, SolrXmlInZkTest, TestFastOutputStream, BJQParserTest, TermVectorComponentDistributedTest, StressHdfsTest, CursorPagingTest, JSONWriterTest, MinimalSchemaTest, AddBlockUpdateTest, TestJmxIntegration, TestPhraseSuggestions, FullSolrCloudDistribCmdsTest, TestCSVLoader, DeleteShardTest, TestCollationFieldDocValues, AnalyticsQueryTest, ReplicationFactorTest, HighlighterConfigTest, NumericFieldsTest, SortByFunctionTest, TestRequestStatusCollectionAPI, CachingDirectoryFactoryTest, XmlUpdateRequestHandlerTest, OpenExchangeRatesOrgProviderTest, TestDFRSimilarityFactory, SystemInfoHandlerTest, TestCSVResponseWriter, DateFieldTest, RangeFacetTest, DocumentAnalysisRequestHandlerTest, TestComplexPhraseQParserPlugin, CollectionsAPIDistributedZkTest, TestRTGBase, AddSchemaFieldsUpdateProcessorFactoryTest, TestManagedResourceStorage, TestCopyFieldCollectionResource, TestAnalyzeInfixSuggestions, IndexBasedSpellCheckerTest, TestFuzzyAnalyzedSuggestions, TestLRUCache, TestOmitPositions, TestCollationField, BlockDirectoryTest, TestStressLucene, TestOverriddenPrefixQueryForCustomFieldType, ChaosMonkeySafeLeaderTest, ConvertedLegacyTest, TestRandomMergePolicy, TestComponentsName, HdfsBasicDistributedZkTest, TestRandomFaceting, DirectSolrSpellCheckerTest, TestQuerySenderNoQuery, LeaderElectionIntegrationTest, PluginInfoTest, TestAnalyzedSuggestions, TestDefaultSearchFieldResource, CurrencyFieldXmlFileTest, TestSearcherReuse, TestLMDirichletSimilarityFactory, TestSchemaSimilarityResource, StatsComponentTest, TestShortCircuitedRequests, BasicDistributedZk2Test, SignatureUpdateProcessorFactoryTest, HdfsUnloadDistributedZkTest, TestDistribDocBasedVersion, ZkSolrClientTest, CursorMarkTest, TestFoldingMultitermQuery, TestManagedStopFilterFactory, CustomCollectionTest]
[junit4] Completed on J1 in 265.98s, 1 test, 1 error <<< FAILURES!
[...truncated 617 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 415 suites, 1697 tests, 1 error, 37 ignored (16 assumptions)
Total time: 76 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0_05 -client -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b15) -
Build # 10528 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10528/
Java: 64bit/jdk1.8.0_20-ea-b15 -XX:+UseCompressedOops -XX:+UseSerialGC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
ERROR: SolrZkClient opens=19 closes=18
Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=19 closes=18
at __randomizedtesting.SeedInfo.seed([96CB2BA00E5C8953]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 1) Thread[id=2857, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-SendThread(127.0.0.1:49448), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 2) Thread[id=2858, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:
1) Thread[id=2857, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-SendThread(127.0.0.1:49448), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
2) Thread[id=2858, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
at __randomizedtesting.SeedInfo.seed([96CB2BA00E5C8953]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2857, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-SendThread(127.0.0.1:49448), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2857, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-SendThread(127.0.0.1:49448), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
at __randomizedtesting.SeedInfo.seed([96CB2BA00E5C8953]:0)
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
deletecollection the collection error [Watcher fired on path: null state: SyncConnected type None]
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: deletecollection the collection error [Watcher fired on path: null state: SyncConnected type None]
at __randomizedtesting.SeedInfo.seed([96CB2BA00E5C8953:172DA5B87903E96F]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:848)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:203)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11162 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/init-core-data-001
[junit4] 2> 565928 T2697 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 565928 T2697 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /jqjx/p
[junit4] 2> 565930 T2697 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 565931 T2697 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 565931 T2698 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 566031 T2697 oasc.ZkTestServer.run start zk server on port:49448
[junit4] 2> 566032 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566112 T2704 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6edc977 name:ZooKeeperConnection Watcher:127.0.0.1:49448 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566113 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 566113 T2697 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 566117 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566118 T2706 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7693bf name:ZooKeeperConnection Watcher:127.0.0.1:49448/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566118 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 566118 T2697 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 566120 T2697 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 566121 T2697 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 566123 T2697 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 566124 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 566124 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 566126 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 566127 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 566128 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 566129 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 566130 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 566131 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 566132 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 566132 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 566134 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 566134 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 566135 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 566135 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 566137 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 566137 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 566138 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 566138 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 566140 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 566140 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 566142 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 566143 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 566145 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566145 T2708 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@724d4ecd name:ZooKeeperConnection Watcher:127.0.0.1:49448/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566146 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> Using new style solr.xml
[junit4] 2> 566147 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566148 T2710 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b9b4c8e name:ZooKeeperConnection Watcher:127.0.0.1:49448 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566148 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 566148 T2697 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 566149 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566150 T2712 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53559cfb name:ZooKeeperConnection Watcher:127.0.0.1:49448/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566150 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 566151 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf2/solrconfig.xml
[junit4] 2> 566151 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
[junit4] 2> 566153 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
[junit4] 2> 566154 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
[junit4] 2> 566155 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 566155 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 566157 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
[junit4] 2> 566157 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
[junit4] 2> 566159 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
[junit4] 2> 566159 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
[junit4] 2> 566160 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
[junit4] 2> 566161 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
[junit4] 2> 566163 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
[junit4] 2> 566163 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
[junit4] 2> 566164 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 566165 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 566166 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
[junit4] 2> 566166 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
[junit4] 2> 566168 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
[junit4] 2> 566168 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
[junit4] 2> 566169 T2697 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
[junit4] 2> 566170 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
[junit4] 2> 566406 T2697 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 566409 T2697 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 566412 T2697 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:45898
[junit4] 2> 566414 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 566414 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 566414 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002
[junit4] 2> 566414 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/'
[junit4] 2> 566424 T2697 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/solr.xml
[junit4] 2> 566431 T2697 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002
[junit4] 2> 566431 T2697 oasc.CoreContainer.<init> New CoreContainer 73032839
[junit4] 2> 566432 T2697 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/]
[junit4] 2> 566432 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 566432 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 566433 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 566433 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 566433 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 566433 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 566433 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 566434 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 566434 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 566436 T2697 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 566436 T2697 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 566436 T2697 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 566437 T2697 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49448/solr
[junit4] 2> 566437 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 566438 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566439 T2723 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ae8574e name:ZooKeeperConnection Watcher:127.0.0.1:49448 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566439 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 566441 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566442 T2725 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c588420 name:ZooKeeperConnection Watcher:127.0.0.1:49448/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566442 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 566443 T2697 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 566445 T2697 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 566446 T2697 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 566447 T2697 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 566449 T2697 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 566451 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 566453 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45898_jqjx%2Fp
[junit4] 2> 566453 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45898_jqjx%2Fp
[junit4] 2> 566455 T2697 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 566456 T2697 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 566458 T2697 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:45898_jqjx%2Fp
[junit4] 2> 566459 T2697 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 566460 T2697 oasc.Overseer.start Overseer (id=91913928463417350-127.0.0.1:45898_jqjx%2Fp-n_0000000000) starting
[junit4] 2> 566461 T2697 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 566475 T2727 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 566475 T2697 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 566477 T2697 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 566478 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 566479 T2726 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 566480 T2697 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002
[junit4] 2> 566486 T2697 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/conf/
[junit4] 2> 566605 T2697 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 566606 T2697 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 566607 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 566609 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566610 T2730 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@426b22fb name:ZooKeeperConnection Watcher:127.0.0.1:49448/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566611 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 566611 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 566613 T2697 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 566833 T2697 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 566834 T2697 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 566837 T2697 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 566838 T2697 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:46741
[junit4] 2> 566840 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 566840 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 566841 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-003
[junit4] 2> 566841 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-003/'
[junit4] 2> 566854 T2697 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-003/solr.xml
[junit4] 2> 566862 T2697 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-003
[junit4] 2> 566862 T2697 oasc.CoreContainer.<init> New CoreContainer 1579497362
[junit4] 2> 566863 T2697 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-003/]
[junit4] 2> 566863 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 566863 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 566863 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 566864 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 566864 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 566864 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 566864 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 566864 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 566865 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 566866 T2697 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 566867 T2697 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 566867 T2697 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 566867 T2697 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49448/solr
[junit4] 2> 566868 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 566869 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566870 T2741 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12a6b0d0 name:ZooKeeperConnection Watcher:127.0.0.1:49448 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566871 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 566872 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 566873 T2743 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7dfd4551 name:ZooKeeperConnection Watcher:127.0.0.1:49448/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 566873 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 566876 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 567878 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46741_jqjx%2Fp
[junit4] 2> 567880 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46741_jqjx%2Fp
[junit4] 2> 567906 T2697 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-003
[junit4] 2> 567921 T2697 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-003/conf/
[junit4] 2> 568058 T2697 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 568059 T2697 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 568059 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 568279 T2697 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 568280 T2697 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 568284 T2697 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 568285 T2697 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:60973
[junit4] 2> 568288 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 568288 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 568289 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-004
[junit4] 2> 568289 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-004/'
[junit4] 2> 568301 T2697 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-004/solr.xml
[junit4] 2> 568308 T2697 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-004
[junit4] 2> 568308 T2697 oasc.CoreContainer.<init> New CoreContainer 714840580
[junit4] 2> 568308 T2697 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-004/]
[junit4] 2> 568309 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 568309 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 568309 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 568309 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 568309 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 568310 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 568310 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 568310 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 568310 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 568312 T2697 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 568312 T2697 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 568312 T2697 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 568313 T2697 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49448/solr
[junit4] 2> 568313 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 568314 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 568315 T2755 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fad943b name:ZooKeeperConnection Watcher:127.0.0.1:49448 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 568316 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 568317 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 568318 T2757 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15b5a9a4 name:ZooKeeperConnection Watcher:127.0.0.1:49448/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 568318 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 568323 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 569324 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60973_jqjx%2Fp
[junit4] 2> 569327 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60973_jqjx%2Fp
[junit4] 2> 569336 T2697 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-004
[junit4] 2> 569348 T2697 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-004/conf/
[junit4] 2> 569469 T2697 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 569470 T2697 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 569470 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 569695 T2697 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 569695 T2697 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 569698 T2697 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 569699 T2697 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:60294
[junit4] 2> 569701 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 569701 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 569701 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005
[junit4] 2> 569701 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/'
[junit4] 2> 569711 T2697 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/solr.xml
[junit4] 2> 569728 T2697 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005
[junit4] 2> 569729 T2697 oasc.CoreContainer.<init> New CoreContainer 248246990
[junit4] 2> 569729 T2697 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/]
[junit4] 2> 569729 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 569730 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 569730 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 569730 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 569730 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 569730 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 569731 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 569731 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 569731 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 569733 T2697 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 569733 T2697 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 569733 T2697 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 569734 T2697 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49448/solr
[junit4] 2> 569734 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 569734 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 569736 T2769 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@413b1ad6 name:ZooKeeperConnection Watcher:127.0.0.1:49448 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 569736 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 569738 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 569739 T2771 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@739e374a name:ZooKeeperConnection Watcher:127.0.0.1:49448/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 569739 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 569743 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 570745 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60294_jqjx%2Fp
[junit4] 2> 570746 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60294_jqjx%2Fp
[junit4] 2> 570751 T2697 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005
[junit4] 2> 570755 T2697 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/conf/
[junit4] 2> 570844 T2697 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 570845 T2697 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 570845 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 571059 T2697 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 571060 T2697 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 571063 T2697 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 571064 T2697 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:57160
[junit4] 2> 571066 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 571066 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 571066 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006
[junit4] 2> 571067 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/'
[junit4] 2> 571080 T2697 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/solr.xml
[junit4] 2> 571086 T2697 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006
[junit4] 2> 571086 T2697 oasc.CoreContainer.<init> New CoreContainer 1709179610
[junit4] 2> 571086 T2697 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/]
[junit4] 2> 571087 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 571087 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 571087 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 571087 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 571087 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 571088 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 571088 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 571088 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 571088 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 571090 T2697 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 571090 T2697 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 571091 T2697 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 571091 T2697 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49448/solr
[junit4] 2> 571091 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 571092 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 571093 T2783 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b0f7db0 name:ZooKeeperConnection Watcher:127.0.0.1:49448 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 571094 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 571095 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 571096 T2785 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a1a8dfc name:ZooKeeperConnection Watcher:127.0.0.1:49448/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 571096 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 571100 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 572102 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57160_jqjx%2Fp
[junit4] 2> 572103 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57160_jqjx%2Fp
[junit4] 2> 572112 T2697 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006
[junit4] 2> 572122 T2697 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/conf/
[junit4] 2> 572218 T2697 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 572219 T2697 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 572219 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 572296 T2736 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&collection.configName=conf1&router.field=myOwnField&name=solrj_collection&router.name=compositeId&action=CREATE&numShards=2&wt=javabin&version=2
[junit4] 2> 572297 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572297 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572297 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572297 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572297 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572298 T2727 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 572299 T2787 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 572300 T2787 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection
[junit4] 2> 572300 T2787 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
[junit4] 2> 572302 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572303 T2726 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 572303 T2726 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
[junit4] 2> 572304 T2726 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
[junit4] 2> 572305 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572306 T2785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572306 T2757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572306 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572306 T2771 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572306 T2743 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572306 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572403 T2787 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection solrj_collection, shardNames [shard1, shard2] , replicationFactor : 2
[junit4] 2> 572403 T2787 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:60294_jqjx%2Fp
[junit4] 2> 572405 T2787 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:45898_jqjx%2Fp
[junit4] 2> 572408 T2787 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:57160_jqjx%2Fp
[junit4] 2> 572411 T2787 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:60973_jqjx%2Fp
[junit4] 2> 572478 T2767 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 572481 T2721 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 572481 T2767 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection
[junit4] 2> 572483 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572483 T2767 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 572484 T2781 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 572484 T2751 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 572485 T2721 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection
[junit4] 2> 572486 T2726 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 572486 T2751 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection
[junit4] 2> 572486 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572486 T2721 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 572487 T2751 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 572487 T2726 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:60294/jqjx/p",
[junit4] 2> "core":"solrj_collection_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60294_jqjx%2Fp",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 572488 T2781 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection
[junit4] 2> 572489 T2781 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 572493 T2726 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45898/jqjx/p",
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:45898_jqjx%2Fp",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 572495 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572495 T2726 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:60973/jqjx/p",
[junit4] 2> "core":"solrj_collection_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60973_jqjx%2Fp",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 572497 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572498 T2726 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:57160/jqjx/p",
[junit4] 2> "core":"solrj_collection_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:57160_jqjx%2Fp",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 572499 T2725 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 572601 T2771 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572601 T2757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572601 T2743 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572601 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572601 T2785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 572601 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 573484 T2767 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
[junit4] 2> 573485 T2767 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 573485 T2767 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 573486 T2767 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 573487 T2721 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
[junit4] 2> 573487 T2767 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 573488 T2767 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/solrj_collection_shard1_replica1/'
[junit4] 2> 573487 T2751 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
[junit4] 2> 573488 T2721 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 573489 T2751 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 573490 T2781 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
[junit4] 2> 573490 T2721 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 573491 T2721 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 573490 T2751 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 573491 T2781 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 573493 T2721 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 573494 T2721 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/solrj_collection_shard1_replica2/'
[junit4] 2> 573493 T2751 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 573494 T2781 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 573496 T2781 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 573496 T2751 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 573497 T2751 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-004/solrj_collection_shard2_replica2/'
[junit4] 2> 573500 T2781 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 573500 T2781 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/solrj_collection_shard2_replica1/'
[junit4] 2> 573541 T2767 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 573550 T2767 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 573550 T2721 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 573551 T2781 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 573551 T2751 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 573551 T2767 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 573567 T2767 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
[junit4] 2> 573570 T2781 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 573572 T2781 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 573574 T2721 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 573575 T2751 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 573575 T2721 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 573575 T2751 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 573577 T2781 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
[junit4] 2> 573581 T2721 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
[junit4] 2> 573581 T2751 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
[junit4] 2> 573741 T2767 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 573745 T2767 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 573746 T2767 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 573747 T2781 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 573750 T2781 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 573751 T2781 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 573752 T2767 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 573754 T2721 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 573756 T2767 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 573757 T2721 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 573758 T2721 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 573758 T2767 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 573759 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 573762 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 573762 T2781 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 573763 T2767 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 573764 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 573764 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 573765 T2767 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using configuration from collection solrj_collection
[junit4] 2> 573765 T2767 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 573766 T2781 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 573766 T2767 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/solrj_collection_shard1_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/solrj_collection_shard1_replica1/data/
[junit4] 2> 573766 T2767 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31e9a734
[junit4] 2> 573767 T2721 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 573767 T2767 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/solrj_collection_shard1_replica1/data
[junit4] 2> 573768 T2781 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 573768 T2767 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/solrj_collection_shard1_replica1/data/index/
[junit4] 2> 573768 T2781 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 573771 T2781 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 573768 T2767 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 573772 T2781 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 573771 T2721 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 573772 T2767 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-005/solrj_collection_shard1_replica1/data/index
[junit4] 2> 573773 T2781 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 573773 T2781 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 573773 T2767 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 573773 T2781 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection
[junit4] 2> 573773 T2751 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 573774 T2721 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 573774 T2767 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@470d1ebc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@514c9dd6),segFN=segments_1,generation=1}
[junit4] 2> 573775 T2767 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 573774 T2781 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 573775 T2721 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 573776 T2721 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 573776 T2781 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/solrj_collection_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/solrj_collection_shard2_replica1/data/
[junit4] 2> 573776 T2721 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 573776 T2781 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31e9a734
[junit4] 2> 573777 T2721 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 573777 T2721 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 573777 T2781 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/solrj_collection_shard2_replica1/data
[junit4] 2> 573778 T2721 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using configuration from collection solrj_collection
[junit4] 2> 573778 T2781 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/solrj_collection_shard2_replica1/data/index/
[junit4] 2> 573778 T2721 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 573778 T2781 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 573779 T2767 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 573779 T2721 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/solrj_collection_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/solrj_collection_shard1_replica2/data/
[junit4] 2> 573779 T2767 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 573779 T2781 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-006/solrj_collection_shard2_replica1/data/index
[junit4] 2> 573780 T2767 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 573779 T2721 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31e9a734
[junit4] 2> 573780 T2767 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 573780 T2781 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 573781 T2721 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/solrj_collection_shard1_replica2/data
[junit4] 2> 573781 T2751 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 573781 T2767 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 573781 T2781 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@37ea7440 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@624e7023),segFN=segments_1,generation=1}
[junit4] 2> 573782 T2781 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 573781 T2721 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/solrj_collection_shard1_replica2/data/index/
[junit4] 2> 573783 T2751 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 573782 T2767 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 573783 T2721 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 573783 T2767 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 573784 T2721 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-96CB2BA00E5C8953-001/tempDir-002/solrj_collection_shard1_replica2/data/index
[junit4] 2> 573784 T2767 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 573784 T2767 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 573784 T2721 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 573785 T2767 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 573785 T2781 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 573786 T2767 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 573786 T2767 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 573785 T2721 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2f2f8d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@62de1379),segFN=segments_1,generation=1}
[junit4] 2> 573787 T2721 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 573786 T2767 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 573786 T2781 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 573787 T2767 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 573788 T2781 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 573788 T2767 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 573788 T2767 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 573789 T2767 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 573790 T2751 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 573791 T2767 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 573788 T2781 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 573790 T2721 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 573800 T2781 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 573800 T2721 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 573800 T2781 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 573801 T2721 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 573801 T2781 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 573801 T2767 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 573801 T2767
[...truncated too long message...]
.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 670213 T2857 oaz.ClientCnxn$SendThread.run WARN Session 0x1468b3b1ff90011 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 670712 T2857 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 670713 T2857 oaz.ClientCnxn$SendThread.run WARN Session 0x1468b3b1ff90011 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 105285 T2696 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=2857, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-SendThread(127.0.0.1:49448), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2> NOTE: test params are: codec=Lucene49: {other_tl1=Pulsing41(freqCutoff=12 minBlockSize=8 maxBlockSize=20), _version_=Pulsing41(freqCutoff=12 minBlockSize=8 maxBlockSize=20), multiDefault=PostingsFormat(name=NestedPulsing), a_t=PostingsFormat(name=NestedPulsing), intDefault=Pulsing41(freqCutoff=12 minBlockSize=8 maxBlockSize=20), a_i1=PostingsFormat(name=NestedPulsing), id=Pulsing41(freqCutoff=19 minBlockSize=8 maxBlockSize=20), text=PostingsFormat(name=FSTPulsing41), range_facet_l=PostingsFormat(name=NestedPulsing), timestamp=PostingsFormat(name=NestedPulsing)}, docValues:{timestamp=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=de_DE, timezone=Asia/Vientiane
[junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation 1.8.0_20-ea (64-bit)/cpus=8,threads=2,free=75631864,total=286994432
[junit4] 2> NOTE: All tests run in this JVM: [NoFacetTest, ZkSolrClientTest, TestInfoStreamLogging, TestManagedResource, DirectUpdateHandlerTest, OverseerStatusTest, UniqFieldsUpdateProcessorFactoryTest, SolrTestCaseJ4Test, SortByFunctionTest, MergeStrategyTest, TestCloudManagedSchemaAddField, TestSolrJ, SolrXmlInZkTest, TestLazyCores, StandardRequestHandlerTest, TestHashPartitioner, TestSolrXMLSerializer, TestPartialUpdateDeduplication, DirectSolrConnectionTest, RequestHandlersTest, DistanceFunctionTest, ModifyConfFileTest, SolrCoreCheckLockOnStartupTest, DateFieldTest, TestZkChroot, ZkNodePropsTest, DeleteReplicaTest, QueryFacetTest, DebugComponentTest, CoreMergeIndexesAdminHandlerTest, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestManagedSchemaFieldResource, PrimUtilsTest, SolrRequestParserTest, TestSolrXmlPersistor, ConvertedLegacyTest, TestStressLucene, TestReRankQParserPlugin, TestDistribDocBasedVersion, ExpressionTest, TestWordDelimiterFilterFactory, TestIndexSearcher, LegacyHTMLStripCharFilterTest, TestComplexPhraseQParserPlugin, TestManagedResourceStorage, TestDistributedGrouping, AddSchemaFieldsUpdateProcessorFactoryTest, NumericFieldsTest, TestLRUCache, BJQParserTest, SolrIndexConfigTest, MultiTermTest, TestRTGBase, TestMultiCoreConfBootstrap, SimplePostToolTest, TestArbitraryIndexDir, TestSolrDeletionPolicy2, HdfsLockFactoryTest, ChaosMonkeySafeLeaderTest, FieldAnalysisRequestHandlerTest, ReplicationFactorTest, ShardRoutingCustomTest, DocExpirationUpdateProcessorFactoryTest, DeleteInactiveReplicaTest, TestSchemaSimilarityResource, CursorMarkTest, TestIndexingPerformance, MinimalSchemaTest, DocValuesMultiTest, TestNonNRTOpen, DefaultValueUpdateProcessorTest, SuggestComponentTest, HighlighterTest, BlockDirectoryTest, TestStressReorder, TestReloadAndDeleteDocs, PluginInfoTest, AnalysisErrorHandlingTest, CoreAdminCreateDiscoverTest, CollectionsAPIDistributedZkTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=96CB2BA00E5C8953 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de_DE -Dtests.timezone=Asia/Vientiane -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ERROR: SolrZkClient opens=19 closes=18
[junit4] > at __randomizedtesting.SeedInfo.seed([96CB2BA00E5C8953]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
[junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:
[junit4] > 1) Thread[id=2857, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-SendThread(127.0.0.1:49448), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 2) Thread[id=2858, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
[junit4] > at __randomizedtesting.SeedInfo.seed([96CB2BA00E5C8953]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=2857, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[96CB2BA00E5C8953]-SendThread(127.0.0.1:49448), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > at __randomizedtesting.SeedInfo.seed([96CB2BA00E5C8953]:0)
[junit4] Completed on J1 in 105.31s, 1 test, 1 failure, 3 errors <<< FAILURES!
[...truncated 732 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 415 suites, 1697 tests, 3 suite-level errors, 1 error, 927 ignored (10 assumptions)
Total time: 54 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0_20-ea-b15 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_05) - Build #
10527 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10527/
Java: 64bit/jdk1.8.0_05 -XX:-UseCompressedOops -XX:+UseSerialGC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ClusterStateUpdateTest
Error Message:
ERROR: SolrZkClient opens=6 closes=5
Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=6 closes=5
at __randomizedtesting.SeedInfo.seed([4AF9B52733B3FC8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ClusterStateUpdateTest
Error Message:
6 threads leaked from SUITE scope at org.apache.solr.cloud.ClusterStateUpdateTest: 1) Thread[id=1057, name=Thread-441, state=WAITING, group=TGRP-ClusterStateUpdateTest] at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1245) at java.lang.Thread.join(Thread.java:1319) at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302) at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:108) at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:227) 2) Thread[id=1061, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-ClusterStateUpdateTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120) 3) Thread[id=1058, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-ClusterStateUpdateTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=1080, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-EventThread, state=WAITING, group=TGRP-ClusterStateUpdateTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 5) Thread[id=1059, name=SessionTracker, state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest] at java.lang.Object.wait(Native Method) at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146) 6) Thread[id=1079, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-SendThread(127.0.0.1:39576), state=RUNNABLE, group=TGRP-ClusterStateUpdateTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.ClusterStateUpdateTest:
1) Thread[id=1057, name=Thread-441, state=WAITING, group=TGRP-ClusterStateUpdateTest]
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1245)
at java.lang.Thread.join(Thread.java:1319)
at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)
at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:108)
at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:227)
2) Thread[id=1061, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-ClusterStateUpdateTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
3) Thread[id=1058, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-ClusterStateUpdateTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=1080, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-EventThread, state=WAITING, group=TGRP-ClusterStateUpdateTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
5) Thread[id=1059, name=SessionTracker, state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]
at java.lang.Object.wait(Native Method)
at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
6) Thread[id=1079, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-SendThread(127.0.0.1:39576), state=RUNNABLE, group=TGRP-ClusterStateUpdateTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
at __randomizedtesting.SeedInfo.seed([4AF9B52733B3FC8]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ClusterStateUpdateTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=1079, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-SendThread(127.0.0.1:39576), state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=1079, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-SendThread(127.0.0.1:39576), state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
at __randomizedtesting.SeedInfo.seed([4AF9B52733B3FC8]:0)
REGRESSION: org.apache.solr.cloud.ClusterStateUpdateTest.testCoreRegistration
Error Message:
Stack Trace:
org.apache.solr.common.cloud.ZooKeeperException:
at org.apache.solr.cloud.ZkController.init(ZkController.java:574)
at org.apache.solr.cloud.ZkController.<init>(ZkController.java:300)
at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:131)
at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:61)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:216)
at org.apache.solr.cloud.ClusterStateUpdateTest.setUp(ClusterStateUpdateTest.java:129)
at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:861)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /live_nodes/127.0.0.1:1662_solr
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:177)
at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:174)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:174)
at org.apache.solr.cloud.ZkController.createEphemeralLiveNode(ZkController.java:701)
at org.apache.solr.cloud.ZkController.init(ZkController.java:544)
... 43 more
Build Log:
[...truncated 10869 lines...]
[junit4] Suite: org.apache.solr.cloud.ClusterStateUpdateTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/init-core-data-001
[junit4] 2> 336861 T1056 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 336866 T1056 oas.SolrTestCaseJ4.setUp ###Starting testCoreRegistration
[junit4] 2> 336866 T1056 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 336867 T1057 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 336967 T1056 oasc.ZkTestServer.run start zk server on port:39576
[junit4] 2> 336967 T1056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 336984 T1063 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7869b110 name:ZooKeeperConnection Watcher:127.0.0.1:39576 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 336985 T1056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 336985 T1056 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 336989 T1056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 336991 T1065 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24747ed8 name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 336991 T1056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 336991 T1056 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 336994 T1056 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 336995 T1056 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 336997 T1056 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 336999 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 337000 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 337016 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 337016 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 337019 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 337019 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 337021 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 337022 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 337023 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 337023 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 337025 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 337025 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 337027 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 337027 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 337029 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 337029 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 337030 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 337030 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 337032 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 337032 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 337034 T1056 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 337034 T1056 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 337039 T1056 oasc.ClusterStateUpdateTest.setUp ####SETUP_START testCoreRegistration
[junit4] 2> 337040 T1056 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/tempDir-001/'
[junit4] 2> 337050 T1056 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/tempDir-001/solr.xml
[junit4] 2> 337058 T1056 oasc.CoreContainer.<init> New CoreContainer 1206295470
[junit4] 2> 337058 T1056 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/tempDir-001/]
[junit4] 2> 337058 T1056 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 337058 T1056 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
[junit4] 2> 337059 T1056 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 337059 T1056 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 337059 T1056 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 337059 T1056 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 337059 T1056 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 337059 T1056 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 337060 T1056 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 337062 T1056 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 337062 T1056 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 337062 T1056 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 337063 T1056 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39576/solr
[junit4] 2> 337063 T1056 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 337063 T1056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 337065 T1067 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1669f2e3 name:ZooKeeperConnection Watcher:127.0.0.1:39576 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 337065 T1056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 337066 T1056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 337067 T1069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1753c0b name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 337068 T1056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 337069 T1056 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 337071 T1056 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 337072 T1056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 337075 T1056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 337086 T1056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 337093 T1056 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 337094 T1056 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:1661_solr
[junit4] 2> 337095 T1056 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:1661_solr
[junit4] 2> 337097 T1056 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 337099 T1056 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 337106 T1056 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:1661_solr
[junit4] 2> 337106 T1056 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 337108 T1056 oasc.Overseer.start Overseer (id=91913365921333251-127.0.0.1:1661_solr-n_0000000000) starting
[junit4] 2> 337110 T1056 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 337116 T1056 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 337117 T1071 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 337118 T1056 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 337121 T1056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 337123 T1070 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 337130 T1072 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 337130 T1072 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 337131 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 337131 T1072 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 337132 T1070 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 337132 T1070 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:1661/solr",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:1661_solr",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":"127.0.0.1:1661_solr_collection1"}
[junit4] 2> 337133 T1070 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 337134 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 337135 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 338132 T1072 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 338132 T1072 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 338133 T1072 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 338133 T1072 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 338134 T1072 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/tempDir-001/collection1/'
[junit4] 2> 338155 T1072 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 338175 T1072 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 338176 T1072 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 338180 T1072 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 338281 T1072 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 338284 T1072 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 338285 T1072 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 338296 T1072 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 338298 T1072 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 338301 T1072 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 338302 T1072 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 338302 T1072 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 338303 T1072 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 338303 T1072 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 338304 T1072 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 338304 T1072 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 338304 T1072 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 338304 T1072 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/tempDir-001/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1/
[junit4] 2> 338305 T1072 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@a621fb0
[junit4] 2> 338305 T1072 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1
[junit4] 2> 338306 T1072 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1/index/
[junit4] 2> 338306 T1072 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1/index' doesn't exist. Creating new index...
[junit4] 2> 338306 T1072 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1/index
[junit4] 2> 338307 T1072 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=42, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 338307 T1072 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@3b7eb889 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65cdbfb3)),segFN=segments_1,generation=1}
[junit4] 2> 338308 T1072 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 338308 T1072 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4] 2> 338311 T1072 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 338311 T1072 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 338311 T1072 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 338311 T1072 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 338312 T1072 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 338312 T1072 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 338312 T1072 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
[junit4] 2> 338312 T1072 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4] 2> 338313 T1072 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 338313 T1072 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 338313 T1072 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 338313 T1072 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 338314 T1072 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 338314 T1072 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 338314 T1072 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 338315 T1072 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 338315 T1072 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4] 2> 338315 T1072 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4] 2> 338316 T1072 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 338316 T1072 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 338316 T1072 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 338317 T1072 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4] 2> 338317 T1072 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4] 2> 338317 T1072 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4] 2> 338317 T1072 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 338318 T1072 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 338318 T1072 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 338318 T1072 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 338318 T1072 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 338319 T1072 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 338319 T1072 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 338319 T1072 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 338320 T1072 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 338320 T1072 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4] 2> 338320 T1072 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4] 2> 338322 T1072 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 338323 T1072 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4] 2> 338325 T1072 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 338325 T1072 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 338325 T1072 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=19.83203125, floorSegmentMB=1.2841796875, forceMergeDeletesPctAllowed=29.44284292161606, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7828449067356973
[junit4] 2> 338326 T1072 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@3b7eb889 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65cdbfb3)),segFN=segments_1,generation=1}
[junit4] 2> 338326 T1072 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 338326 T1072 oass.SolrIndexSearcher.<init> Opening Searcher@2bbbd2b0[collection1] main
[junit4] 2> 338330 T1072 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 338331 T1072 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 338331 T1072 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
[junit4] 2> 338331 T1072 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 338331 T1072 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 338332 T1072 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 338332 T1072 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 338332 T1072 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 338342 T1072 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 338368 T1073 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2bbbd2b0[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 338370 T1072 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 338370 T1076 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:1661/solr collection:collection1 shard:shard1
[junit4] 2> 338371 T1056 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/tempDir-001/'
[junit4] 2> 338371 T1076 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 338375 T1076 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 338377 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 338377 T1076 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 338377 T1076 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 338378 T1076 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:1661/solr/collection1/ shard1
[junit4] 2> 338378 T1070 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 338378 T1076 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 338379 T1056 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/tempDir-001/solr.xml
[junit4] 2> 338380 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 338382 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 338391 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 338394 T1056 oasc.CoreContainer.<init> New CoreContainer 1207300552
[junit4] 2> 338394 T1056 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/tempDir-001/]
[junit4] 2> 338395 T1056 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 338395 T1056 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
[junit4] 2> 338395 T1056 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 338395 T1056 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 338395 T1056 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 338396 T1056 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 338396 T1056 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 338396 T1056 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 338396 T1056 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 338398 T1056 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 338398 T1056 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 338398 T1056 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 338398 T1056 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39576/solr
[junit4] 2> 338398 T1056 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 338399 T1056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 338401 T1078 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3225b01f name:ZooKeeperConnection Watcher:127.0.0.1:39576 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 338402 T1056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 338404 T1056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 338405 T1080 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e160712 name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 338405 T1056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 338409 T1056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 338493 T1080 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 338493 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 338540 T1076 oasc.ZkController.register We are http://127.0.0.1:1661/solr/collection1/ and leader is http://127.0.0.1:1661/solr/collection1/
[junit4] 2> 338541 T1076 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:1661/solr
[junit4] 2> 338541 T1076 oasc.ZkController.checkRecovery WARN Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4] 2> 338541 T1076 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 338541 T1076 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 338543 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 338543 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 338543 T1076 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 338543 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 338544 T1070 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 338544 T1070 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:1661/solr",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:1661_solr",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":"127.0.0.1:1661_solr_collection1"}
[junit4] 2> 338546 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 338647 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 338647 T1080 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 339411 T1056 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:1662_solr
[junit4] 2> 341122 T1058 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1468ab825da0003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 341222 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 341222 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 341223 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 341223 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 341223 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 341223 T1069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1753c0b name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 341223 T1069 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 341223 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 341412 T1058 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1468ab825da0005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 341512 T1080 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e160712 name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 341512 T1080 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 343093 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 343094 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 343094 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 343094 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 343094 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 343094 T1069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1753c0b name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 343094 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 343281 T1080 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e160712 name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 345094 T1058 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1468ab825da0003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 345194 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 345194 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 345194 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 345194 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 345195 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 345195 T1069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1753c0b name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 345195 T1069 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 345195 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 345282 T1058 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1468ab825da0005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 345334 T1060 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 5921ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 345334 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 345335 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:169)
[junit4] 2>
[junit4] 2> 345335 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:169)
[junit4] 2>
[junit4] 2> 345336 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:169)
[junit4] 2>
[junit4] 2> 345337 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:169)
[junit4] 2>
[junit4] 2> 345337 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:169)
[junit4] 2>
[junit4] 2> 345338 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:169)
[junit4] 2>
[junit4] 2> 345356 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 345356 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 345357 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 345383 T1080 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e160712 name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 345383 T1080 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 346451 T1080 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e160712 name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 346563 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 346563 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 346564 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 346564 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 346564 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 346564 T1069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1753c0b name:ZooKeeperConnection Watcher:127.0.0.1:39576/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 346564 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 346694 T1071 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4] 2> 346883 T1056 oasc.ZkController.init ERROR org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /live_nodes/127.0.0.1:1662_solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:177)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:174)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:174)
[junit4] 2> at org.apache.solr.cloud.ZkController.createEphemeralLiveNode(ZkController.java:701)
[junit4] 2> at org.apache.solr.cloud.ZkController.init(ZkController.java:544)
[junit4] 2> at org.apache.solr.cloud.ZkController.<init>(ZkController.java:300)
[junit4] 2> at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:131)
[junit4] 2> at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:61)
[junit4] 2> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:216)
[junit4] 2> at org.apache.solr.cloud.ClusterStateUpdateTest.setUp(ClusterStateUpdateTest.java:129)
[junit4] 2> at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:483)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:861)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 346884 T1056 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1206295470
[junit4] 2> 346884 T1056 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 346885 T1056 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 346885 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
[junit4] 2> at org.apache.zookeeper.server.DataTree.createNode(DataTree.java:523)
[junit4] 2> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:791)
[junit4] 2> at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
[junit4] 2> at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 346885 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 346886 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 346887 T1056 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@ef76ba8
[junit4] 2> 346887 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 346887 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 346887 T1070 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 346888 T1070 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:1661/solr",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:1661_solr",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":"127.0.0.1:1661_solr_collection1"}
[junit4] 2> 346889 T1069 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 346890 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
[junit4] 2> at org.apache.zookeeper.server.DataTree.setData(DataTree.java:620)
[junit4] 2> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:807)
[junit4] 2> at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
[junit4] 2> at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 346890 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 346890 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
[junit4] 2> at org.apache.zookeeper.server.DataTree.setData(DataTree.java:620)
[junit4] 2> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:807)
[junit4] 2> at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
[junit4] 2> at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 346891 T1080 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 346896 T1056 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 346896 T1056 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 346896 T1056 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 346897 T1056 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 346898 T1056 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 346899 T1056 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 346899 T1056 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1;done=false>>]
[junit4] 2> 346899 T1056 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1
[junit4] 2> 346899 T1056 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1/index;done=false>>]
[junit4] 2> 346899 T1056 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.ClusterStateUpdateTest-4AF9B52733B3FC8-001/zkData-001/data1/index
[junit4] 2> 346900 T1070 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:1661_solr
[junit4] 2> 346902 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
[junit4] 2> at org.apache.zookeeper.server.DataTree.deleteNode(DataTree.java:594)
[junit4] 2> at org.apache.zookeeper.server.DataTree.killSession(DataTree.java:966)
[junit4] 2> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:818)
[junit4] 2> at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
[junit4] 2> at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 346902 T1060 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
[junit4] 2> at org.apache.zookeeper.server.DataTree.deleteNode(DataTree.java:594)
[junit4] 2> at org.apache.zookeeper.server.DataTree.killSession(DataTree.java:966)
[junit4] 2> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:818)
[junit4] 2> at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
[junit4] 2> at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 346903 T1056 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1207300552
[junit4] 2> 346904 T1056 oasc.CoreContainer.shutdown WARN Error shutting down CoreAdminHandler. Continuing to shutdown CoreContainer.
[junit4] 2> java.lang.NullPointerException
[junit4] 2> at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:375)
[junit4] 2> at org.apache.solr.cloud.ClusterStateUpdateTest.tearDown(ClusterStateUpdateTest.java:258)
[junit4] 2> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:483)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ClusterStateUpdateTest -Dtests.method=testCoreRegistration -Dtests.seed=4AF9B52733B3FC8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=no -Dtests.timezone=US/Alaska -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 10.1s J0 | ClusterStateUpdateTest.testCoreRegistration <<<
[junit4] > Throwable #1: org.apache.solr.common.cloud.ZooKeeperException:
[junit4] > at org.apache.solr.cloud.ZkController.init(ZkController.java:574)
[junit4] > at org.apache.solr.cloud.ZkController.<init>(ZkController.java:300)
[junit4] > at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:131)
[junit4] > at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:61)
[junit4] > at org.apache.solr.core.CoreContainer.load(CoreContainer.java:216)
[junit4] > at org.apache.solr.cloud.ClusterStateUpdateTest.setUp(ClusterStateUpdateTest.java:129)
[
[...truncated too long message...]
79 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 369597 T1079 oaz.ClientCnxn$SendThread.run WARN Session 0x1468ab825da0005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 33236 T1055 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=1079, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-SendThread(127.0.0.1:39576), state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=no, timezone=US/Alaska
[junit4] 2> NOTE: Linux 3.8.0-41-generic amd64/Oracle Corporation 1.8.0_05 (64-bit)/cpus=8,threads=3,free=87776048,total=200462336
[junit4] 2> NOTE: All tests run in this JVM: [HdfsCollectionsAPIDistributedZkTest, SyncSliceTest, SolrIndexSplitterTest, ConnectionManagerTest, DistributedQueryElevationComponentTest, PolyFieldTest, DocValuesTest, QueryElevationComponentTest, BasicDistributedZkTest, TestManagedSynonymFilterFactory, SchemaVersionSpecificBehaviorTest, FastVectorHighlighterTest, TestSearchPerf, TestPHPSerializedResponseWriter, TestGroupingSearch, BinaryUpdateRequestHandlerTest, TestSolrIndexConfig, SolrInfoMBeanTest, DistribDocExpirationUpdateProcessorTest, TestExceedMaxTermLength, BadComponentTest, QueryParsingTest, TestPerFieldSimilarity, DateRangeFieldTest, DistributedDebugComponentTest, TestPostingsSolrHighlighter, BadCopyFieldTest, HdfsBasicDistributedZk2Test, ChangedSchemaMergeTest, TestNRTOpen, TestBadConfig, TestRealTimeGet, PreAnalyzedUpdateProcessorTest, ClusterStateUpdateTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ClusterStateUpdateTest -Dtests.seed=4AF9B52733B3FC8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=no -Dtests.timezone=US/Alaska -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | ClusterStateUpdateTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ERROR: SolrZkClient opens=6 closes=5
[junit4] > at __randomizedtesting.SeedInfo.seed([4AF9B52733B3FC8]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
[junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.ClusterStateUpdateTest:
[junit4] > 1) Thread[id=1057, name=Thread-441, state=WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at java.lang.Thread.join(Thread.java:1245)
[junit4] > at java.lang.Thread.join(Thread.java:1319)
[junit4] > at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)
[junit4] > at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:108)
[junit4] > at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:227)
[junit4] > 2) Thread[id=1061, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
[junit4] > 3) Thread[id=1058, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-ClusterStateUpdateTest]
[junit4] > at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4] > at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
[junit4] > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=1080, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-EventThread, state=WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
[junit4] > 5) Thread[id=1059, name=SessionTracker, state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
[junit4] > 6) Thread[id=1079, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-SendThread(127.0.0.1:39576), state=RUNNABLE, group=TGRP-ClusterStateUpdateTest]
[junit4] > at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4] > at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
[junit4] > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] > at __randomizedtesting.SeedInfo.seed([4AF9B52733B3FC8]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=1079, name=TEST-ClusterStateUpdateTest.testCoreRegistration-seed#[4AF9B52733B3FC8]-SendThread(127.0.0.1:39576), state=TIMED_WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > at __randomizedtesting.SeedInfo.seed([4AF9B52733B3FC8]:0)
[junit4] Completed on J0 in 33.27s, 1 test, 1 failure, 3 errors <<< FAILURES!
[...truncated 1041 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 415 suites, 1697 tests, 3 suite-level errors, 1 error, 1182 ignored (9 assumptions)
Total time: 58 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0_05 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20-ea-b15) -
Build # 10526 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10526/
Java: 32bit/jdk1.8.0_20-ea-b15 -client -XX:+UseConcMarkSweepGC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
ERROR: SolrZkClient opens=20 closes=19
Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=20 closes=19
at __randomizedtesting.SeedInfo.seed([59522161B48C241B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 1) Thread[id=254, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-SendThread(127.0.0.1:55427), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 2) Thread[id=255, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:
1) Thread[id=254, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-SendThread(127.0.0.1:55427), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
2) Thread[id=255, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
at __randomizedtesting.SeedInfo.seed([59522161B48C241B]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=254, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-SendThread(127.0.0.1:55427), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=254, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-SendThread(127.0.0.1:55427), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
at __randomizedtesting.SeedInfo.seed([59522161B48C241B]:0)
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
some core start times did not change on reload
Stack Trace:
java.lang.AssertionError: some core start times did not change on reload
at __randomizedtesting.SeedInfo.seed([59522161B48C241B:D8B4AF79C3D34427]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:836)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:203)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 10791 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/init-core-data-001
[junit4] 2> 212180 T94 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 212181 T94 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 212185 T94 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 212189 T94 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 212192 T95 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 212291 T94 oasc.ZkTestServer.run start zk server on port:55427
[junit4] 2> 212344 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212454 T101 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13662f1 name:ZooKeeperConnection Watcher:127.0.0.1:55427 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212455 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212458 T94 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 212501 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212505 T103 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@147b975 name:ZooKeeperConnection Watcher:127.0.0.1:55427/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212506 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212510 T94 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 212519 T94 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 212526 T94 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 212532 T94 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 212542 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 212543 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 212550 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 212551 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 212557 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 212558 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 212564 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 212565 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 212569 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 212569 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 212576 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 212577 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 212584 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 212585 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 212592 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 212593 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 212600 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 212601 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 212608 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 212609 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 212615 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 212616 T94 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> Using old style solr.xml
[junit4] 2> 212626 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212629 T105 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d3d28 name:ZooKeeperConnection Watcher:127.0.0.1:55427 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212630 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212631 T94 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 212644 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 212645 T107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b0f306 name:ZooKeeperConnection Watcher:127.0.0.1:55427/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 212646 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 212647 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf2/solrconfig.xml
[junit4] 2> 212648 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
[junit4] 2> 212654 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
[junit4] 2> 212655 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
[junit4] 2> 212663 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 212664 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 212668 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
[junit4] 2> 212668 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
[junit4] 2> 212672 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
[junit4] 2> 212672 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
[junit4] 2> 212678 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
[junit4] 2> 212679 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
[junit4] 2> 212683 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
[junit4] 2> 212684 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
[junit4] 2> 212695 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 212696 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 212702 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
[junit4] 2> 212703 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
[junit4] 2> 212708 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
[junit4] 2> 212709 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
[junit4] 2> 212721 T94 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
[junit4] 2> 212722 T94 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
[junit4] 2> 212775 T3 oasc.CoreContainer.finalize ERROR CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=13047865
[junit4] 2> 213107 T94 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 213122 T94 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58317
[junit4] 2> 213124 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 213125 T94 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 213126 T94 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002
[junit4] 2> 213128 T94 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/'
[junit4] 2> 213152 T94 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/solr.xml
[junit4] 2> 213173 T94 oasc.CoreContainer.<init> New CoreContainer 14661345
[junit4] 2> 213174 T94 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/]
[junit4] 2> 213177 T94 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 213177 T94 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 213178 T94 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 213178 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 213178 T94 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 213179 T94 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 213179 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 213180 T94 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 213180 T94 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 213181 T94 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 213181 T94 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 213182 T94 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 213183 T94 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55427/solr
[junit4] 2> 213198 T94 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 213200 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 213202 T118 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1720b45 name:ZooKeeperConnection Watcher:127.0.0.1:55427 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 213203 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 213210 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 213212 T120 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17c0731 name:ZooKeeperConnection Watcher:127.0.0.1:55427/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 213213 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 213222 T94 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 213233 T94 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 213241 T94 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 213248 T94 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 213256 T94 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 213269 T94 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 213273 T94 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58317_
[junit4] 2> 213279 T94 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58317_
[junit4] 2> 213286 T94 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 213292 T94 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 213311 T94 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:58317_
[junit4] 2> 213312 T94 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 213317 T94 oasc.Overseer.start Overseer (id=91912831065915397-127.0.0.1:58317_-n_0000000000) starting
[junit4] 2> 213337 T94 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 213384 T122 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 213385 T94 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 213390 T94 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 213400 T94 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 213417 T121 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 213420 T94 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 213420 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 213425 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 213427 T125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c9cc9 name:ZooKeeperConnection Watcher:127.0.0.1:55427/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 213427 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 213429 T94 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 213433 T94 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 213731 T94 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 213732 T94 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 213735 T94 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:46995
[junit4] 2> 213738 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 213738 T94 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 213739 T94 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003
[junit4] 2> 213739 T94 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/'
[junit4] 2> 213763 T94 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/solr.xml
[junit4] 2> 213780 T94 oasc.CoreContainer.<init> New CoreContainer 6439974
[junit4] 2> 213780 T94 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/]
[junit4] 2> 213781 T94 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 213781 T94 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 213782 T94 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 213782 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 213782 T94 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 213783 T94 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 213783 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 213784 T94 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 213784 T94 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 213785 T94 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 213785 T94 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 213786 T94 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 213786 T94 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55427/solr
[junit4] 2> 213786 T94 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 213787 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 213791 T136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11796ab name:ZooKeeperConnection Watcher:127.0.0.1:55427 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 213792 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 213799 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 213801 T138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@97aba9 name:ZooKeeperConnection Watcher:127.0.0.1:55427/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 213801 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 213814 T94 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 214819 T94 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46995_
[junit4] 2> 214821 T94 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46995_
[junit4] 2> 214833 T94 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 214833 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 215109 T94 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 215110 T94 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 215112 T94 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48823
[junit4] 2> 215114 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 215114 T94 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 215115 T94 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004
[junit4] 2> 215115 T94 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/'
[junit4] 2> 215141 T94 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/solr.xml
[junit4] 2> 215160 T94 oasc.CoreContainer.<init> New CoreContainer 190993
[junit4] 2> 215161 T94 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/]
[junit4] 2> 215162 T94 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 215162 T94 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 215162 T94 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 215163 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 215163 T94 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 215163 T94 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 215164 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 215164 T94 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 215165 T94 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 215165 T94 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 215166 T94 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 215166 T94 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 215166 T94 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55427/solr
[junit4] 2> 215167 T94 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 215167 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 215170 T150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f0c902 name:ZooKeeperConnection Watcher:127.0.0.1:55427 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 215171 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 215174 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 215175 T152 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b9d6e name:ZooKeeperConnection Watcher:127.0.0.1:55427/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 215176 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 215182 T94 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 216186 T94 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48823_
[junit4] 2> 216188 T94 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48823_
[junit4] 2> 216205 T94 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 216206 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 216460 T94 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 216460 T94 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 216463 T94 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:38833
[junit4] 2> 216464 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 216465 T94 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 216465 T94 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-005
[junit4] 2> 216466 T94 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-005/'
[junit4] 2> 216484 T94 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-005/solr.xml
[junit4] 2> 216500 T94 oasc.CoreContainer.<init> New CoreContainer 11255043
[junit4] 2> 216501 T94 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-005/]
[junit4] 2> 216501 T94 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 216502 T94 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 216502 T94 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 216502 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 216503 T94 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 216503 T94 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 216506 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 216506 T94 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 216506 T94 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 216507 T94 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 216508 T94 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 216508 T94 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 216509 T94 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55427/solr
[junit4] 2> 216509 T94 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 216510 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 216512 T164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b7c4a4 name:ZooKeeperConnection Watcher:127.0.0.1:55427 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 216513 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 216516 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 216517 T166 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2834e5 name:ZooKeeperConnection Watcher:127.0.0.1:55427/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 216517 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 216524 T94 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 217528 T94 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38833_
[junit4] 2> 217532 T94 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38833_
[junit4] 2> 217548 T94 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 217549 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 217850 T94 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 217851 T94 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 217855 T94 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44089
[junit4] 2> 217857 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 217858 T94 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 217858 T94 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006
[junit4] 2> 217859 T94 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/'
[junit4] 2> 217876 T94 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/solr.xml
[junit4] 2> 217904 T94 oasc.CoreContainer.<init> New CoreContainer 11483212
[junit4] 2> 217904 T94 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/]
[junit4] 2> 217905 T94 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 217906 T94 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 217906 T94 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 217907 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 217907 T94 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 217908 T94 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 217909 T94 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 217909 T94 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 217910 T94 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 217910 T94 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 217911 T94 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 217911 T94 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 217912 T94 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55427/solr
[junit4] 2> 217912 T94 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 217913 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 217915 T178 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d39ac6 name:ZooKeeperConnection Watcher:127.0.0.1:55427 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 217916 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 217919 T94 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 217921 T180 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a89026 name:ZooKeeperConnection Watcher:127.0.0.1:55427/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 217921 T94 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 217928 T94 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 218931 T94 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44089_
[junit4] 2> 218934 T94 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44089_
[junit4] 2> 218949 T94 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 218950 T94 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 218971 T132 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&collection.configName=conf1&router.field=myOwnField&name=solrj_collection&router.name=compositeId&action=CREATE&numShards=2&wt=javabin&version=2
[junit4] 2> 218975 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 218975 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 218976 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 218976 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 218977 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 218981 T122 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 218986 T182 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 218989 T182 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection
[junit4] 2> 218989 T182 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
[junit4] 2> 218999 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 219003 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 219005 T121 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
[junit4] 2> 219005 T121 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
[junit4] 2> 219015 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 219016 T120 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219017 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219016 T125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219016 T152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219016 T138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219017 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219100 T182 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection solrj_collection, shardNames [shard1, shard2] , replicationFactor : 2
[junit4] 2> 219100 T182 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:44089_
[junit4] 2> 219105 T182 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:58317_
[junit4] 2> 219106 T182 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:46995_
[junit4] 2> 219108 T182 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:48823_
[junit4] 2> 219115 T173 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 219115 T114 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 219115 T148 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 219115 T134 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 219120 T134 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection
[junit4] 2> 219120 T148 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection
[junit4] 2> 219120 T114 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection
[junit4] 2> 219121 T173 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection
[junit4] 2> 219122 T134 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 219122 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 219123 T114 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 219122 T148 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 219123 T173 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 219124 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 219125 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:46995",
[junit4] 2> "core":"solrj_collection_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:46995_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 219129 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 219130 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48823",
[junit4] 2> "core":"solrj_collection_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:48823_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 219134 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 219135 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58317",
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:58317_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 219137 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 219138 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44089",
[junit4] 2> "core":"solrj_collection_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:44089_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 219140 T120 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 219244 T138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219244 T120 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219244 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219244 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219244 T152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 219244 T125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 220123 T134 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
[junit4] 2> 220123 T114 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
[junit4] 2> 220124 T148 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
[junit4] 2> 220124 T114 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/solr.xml
[junit4] 2> 220124 T148 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/solr.xml
[junit4] 2> 220125 T114 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 220124 T173 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
[junit4] 2> 220124 T134 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/solr.xml
[junit4] 2> 220126 T114 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 220125 T148 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 220126 T114 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 220126 T173 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/solr.xml
[junit4] 2> 220126 T134 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 220131 T114 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 220131 T173 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 220131 T148 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 220132 T114 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/solrj_collection_shard1_replica2/'
[junit4] 2> 220132 T134 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 220132 T173 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 220133 T173 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 220132 T148 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 220133 T134 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 220134 T173 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 220135 T148 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 220135 T134 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 220135 T173 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/solrj_collection_shard1_replica1/'
[junit4] 2> 220136 T134 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/solrj_collection_shard2_replica1/'
[junit4] 2> 220135 T148 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/solrj_collection_shard2_replica2/'
[junit4] 2> 220224 T148 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 220238 T114 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 220255 T173 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 220258 T134 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 220290 T148 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 220293 T148 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 220310 T114 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 220313 T114 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 220320 T173 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 220322 T173 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 220323 T134 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 220324 T134 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 220364 T114 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
[junit4] 2> 220379 T148 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
[junit4] 2> 220392 T134 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
[junit4] 2> 220399 T173 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
[junit4] 2> 221221 T134 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 221235 T114 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 221250 T134 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 221263 T134 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 221264 T114 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 221266 T148 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 221267 T114 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 221293 T148 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 221298 T148 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 221311 T134 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 221317 T173 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 221324 T134 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 221330 T173 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 221331 T114 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 221333 T173 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 221335 T134 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 221338 T134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 221339 T114 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 221341 T134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 221343 T134 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 221344 T134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 221345 T134 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 221345 T148 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 221346 T134 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection
[junit4] 2> 221349 T134 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 221351 T114 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 221351 T134 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/solrj_collection_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/solrj_collection_shard2_replica1/data/
[junit4] 2> 221353 T134 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1934f6b
[junit4] 2> 221353 T114 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 221354 T114 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 221355 T114 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 221355 T148 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 221357 T114 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 221357 T114 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 221358 T114 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using configuration from collection solrj_collection
[junit4] 2> 221359 T114 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 221360 T114 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/solrj_collection_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/solrj_collection_shard1_replica2/data/
[junit4] 2> 221361 T114 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1934f6b
[junit4] 2> 221364 T148 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 221367 T148 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 221367 T173 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 221367 T148 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 221369 T148 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 221371 T148 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 221372 T148 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 221373 T148 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using configuration from collection solrj_collection
[junit4] 2> 221373 T173 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 221373 T148 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 221377 T148 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/solrj_collection_shard2_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/solrj_collection_shard2_replica2/data/
[junit4] 2> 221378 T148 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1934f6b
[junit4] 2> 221381 T173 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 221382 T173 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 221383 T173 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 221383 T173 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 221384 T173 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 221385 T173 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 221385 T173 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using configuration from collection solrj_collection
[junit4] 2> 221386 T173 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 221387 T173 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/solrj_collection_shard1_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/solrj_collection_shard1_replica1/data/
[junit4] 2> 221387 T173 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1934f6b
[junit4] 2> 221391 T173 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/solrj_collection_shard1_replica1/data
[junit4] 2> 221392 T148 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/solrj_collection_shard2_replica2/data
[junit4] 2> 221391 T114 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/solrj_collection_shard1_replica2/data
[junit4] 2> 221393 T148 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/solrj_collection_shard2_replica2/data/index/
[junit4] 2> 221392 T134 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/solrj_collection_shard2_replica1/data
[junit4] 2> 221392 T173 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/solrj_collection_shard1_replica1/data/index/
[junit4] 2> 221395 T134 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/solrj_collection_shard2_replica1/data/index/
[junit4] 2> 221394 T148 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 221394 T114 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/solrj_collection_shard1_replica2/data/index/
[junit4] 2> 221398 T148 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-004/solrj_collection_shard2_replica2/data/index
[junit4] 2> 221397 T134 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 221396 T173 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 221400 T134 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-003/solrj_collection_shard2_replica1/data/index
[junit4] 2> 221400 T148 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=13, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 221398 T114 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 221402 T134 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=13, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 221401 T173 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-006/solrj_collection_shard1_replica1/data/index
[junit4] 2> 221404 T114 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.CollectionsAPIDistributedZkTest-59522161B48C241B-001/tempDir-002/solrj_collection_shard1_replica2/data/index
[junit4] 2> 221404 T148 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@cce416 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@111bc8a),segFN=segments_1,generation=1}
[junit4] 2> 221407 T148 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 221407 T114 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=13, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 221406 T134 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1d9a7af lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@199137),segFN=segments_1,generation=1}
[junit4] 2> 221410 T134 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 221406 T173 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=13, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 221410 T114 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@184b229 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1014e82),segFN=segments_1,generation=1}
[junit4] 2> 221412 T173 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@10ff6ba lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b6b440),segFN=segments_1,generation=1}
[junit4] 2> 221413 T173 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 221412 T114 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 221427 T173 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 221428 T148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 221427 T134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 221429 T148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 221428 T114 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 221428 T173 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 221431 T114 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 221430 T148 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 221430 T134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 221433 T148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 221432 T114 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 221432 T173 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 221435 T114 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 221434 T148 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 221433 T134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 221437 T148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 221436 T114 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 221439 T148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 221435 T173 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 221440 T148 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 221439 T114 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 221437 T134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 221441 T148 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 221440 T173 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 221442 T134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 221442 T114 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 221445 T148 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 221445 T114 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 221444 T134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 221443 T173 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 221447 T148 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 221450 T148 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 221447 T173 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 221451 T148 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 221447 T134 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 221446 T114 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 221452 T134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 221453 T148 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 221454 T114 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 221451 T173 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 221455 T173 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 221454 T148 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 221453 T134 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcess
[...truncated too long message...]
okeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 336173 T254 oaz.ClientCnxn$SendThread.run WARN Session 0x1468a3b9dfc0010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 336672 T254 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 336673 T254 oaz.ClientCnxn$SendThread.run WARN Session 0x1468a3b9dfc0010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 337172 T254 oaz.ClientCnxn$SendThread.run WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] 2>
[junit4] 2> 337672 T254 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 337673 T254 oaz.ClientCnxn$SendThread.run WARN Session 0x1468a3b9dfc0010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 338172 T254 oaz.ClientCnxn$SendThread.run WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] 2>
[junit4] 2> 126494 T93 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=254, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-SendThread(127.0.0.1:55427), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2> NOTE: test params are: codec=Lucene49: {other_tl1=PostingsFormat(name=FSTOrdPulsing41), _version_=PostingsFormat(name=SimpleText), multiDefault=PostingsFormat(name=FSTOrdPulsing41), a_t=PostingsFormat(name=FSTOrdPulsing41), intDefault=PostingsFormat(name=FSTOrdPulsing41), a_i1=PostingsFormat(name=FSTOrdPulsing41), id=TestBloomFilteredLucene41Postings(BloomFilteringPostingsFormat(Lucene41(blocksize=128))), text=PostingsFormat(name=Memory doPackFST= false), range_facet_l=PostingsFormat(name=FSTOrdPulsing41), timestamp=PostingsFormat(name=FSTOrdPulsing41)}, docValues:{timestamp=DocValuesFormat(name=Disk)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ms_MY, timezone=HST
[junit4] 2> NOTE: Linux 3.8.0-41-generic i386/Oracle Corporation 1.8.0_20-ea (32-bit)/cpus=8,threads=2,free=15783456,total=57696256
[junit4] 2> NOTE: All tests run in this JVM: [SolrCmdDistributorTest, TestSolrXmlPersistor, TestSolrDeletionPolicy2, TestAddFieldRealTimeGet, CollectionsAPIDistributedZkTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=59522161B48C241B -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ms_MY -Dtests.timezone=HST -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ERROR: SolrZkClient opens=20 closes=19
[junit4] > at __randomizedtesting.SeedInfo.seed([59522161B48C241B]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
[junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:
[junit4] > 1) Thread[id=254, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-SendThread(127.0.0.1:55427), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 2) Thread[id=255, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
[junit4] > at __randomizedtesting.SeedInfo.seed([59522161B48C241B]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=254, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[59522161B48C241B]-SendThread(127.0.0.1:55427), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > at __randomizedtesting.SeedInfo.seed([59522161B48C241B]:0)
[junit4] Completed on J1 in 126.58s, 1 test, 2 failures, 2 errors <<< FAILURES!
[...truncated 1100 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 415 suites, 1697 tests, 3 suite-level errors, 1 failure, 1221 ignored (9 assumptions)
Total time: 54 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0_20-ea-b15 -client -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any