You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/01/25 11:03:49 UTC

[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3676 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3676/

2 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([AD8248BC50D3AF8D]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([AD8248BC50D3AF8D]:0)




Build Log:
[...truncated 9494 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T362 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /e_li/w
[junit4:junit4]   2> 7 T362 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359100987124
[junit4:junit4]   2> 8 T362 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 21 T363 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 24 T363 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 25 T363 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 25 T363 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 25 T363 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 27 T363 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 122 T362 oasc.ZkTestServer.run start zk server on port:65249
[junit4:junit4]   2> 123 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@209f43e6
[junit4:junit4]   2> 124 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 124 T368 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 125 T368 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 125 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65250
[junit4:junit4]   2> 126 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65250
[junit4:junit4]   2> 126 T366 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 130 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080000 with negotiated timeout 10000 for client /140.211.11.196:65250
[junit4:junit4]   2> 130 T368 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080000, negotiated timeout = 10000
[junit4:junit4]   2> 131 T369 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@209f43e6 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 131 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 132 T362 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 136 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080000
[junit4:junit4]   2> 137 T369 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 138 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65250 which had sessionid 0x13c70bb47080000
[junit4:junit4]   2> 137 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080000 closed
[junit4:junit4]   2> 138 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@44f066ab
[junit4:junit4]   2> 140 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 140 T370 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 141 T370 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 141 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65251
[junit4:junit4]   2> 141 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65251
[junit4:junit4]   2> 143 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080001 with negotiated timeout 10000 for client /140.211.11.196:65251
[junit4:junit4]   2> 143 T370 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080001, negotiated timeout = 10000
[junit4:junit4]   2> 144 T371 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44f066ab name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 144 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 145 T362 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 150 T362 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 159 T362 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 164 T362 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 168 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 170 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 188 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 189 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 321 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 322 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 329 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 330 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 335 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 337 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 343 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 345 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 350 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 351 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 357 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 358 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 363 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 364 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 372 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080001
[junit4:junit4]   2> 373 T371 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 373 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65251 which had sessionid 0x13c70bb47080001
[junit4:junit4]   2> 373 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080001 closed
[junit4:junit4]   2> 524 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 529 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65252
[junit4:junit4]   2> 529 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 530 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 530 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491
[junit4:junit4]   2> 531 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/solr.xml
[junit4:junit4]   2> 531 T362 oasc.CoreContainer.<init> New CoreContainer 581217075
[junit4:junit4]   2> 532 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/'
[junit4:junit4]   2> 533 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/'
[junit4:junit4]   2> 588 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 588 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 589 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 589 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 590 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 590 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 591 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 591 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 592 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 592 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 618 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 638 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 638 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 639 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@46f5c7a4
[junit4:junit4]   2> 640 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 640 T381 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 641 T381 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 642 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65254
[junit4:junit4]   2> 642 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65254
[junit4:junit4]   2> 645 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080002 with negotiated timeout 20000 for client /140.211.11.196:65254
[junit4:junit4]   2> 645 T381 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080002, negotiated timeout = 20000
[junit4:junit4]   2> 646 T382 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46f5c7a4 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 646 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 648 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080002
[junit4:junit4]   2> 649 T382 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 650 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65254 which had sessionid 0x13c70bb47080002
[junit4:junit4]   2> 649 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080002 closed
[junit4:junit4]   2> 651 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 663 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6bf90542
[junit4:junit4]   2> 665 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 665 T383 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 666 T383 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 666 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65255
[junit4:junit4]   2> 667 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65255
[junit4:junit4]   2> 669 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080003 with negotiated timeout 20000 for client /140.211.11.196:65255
[junit4:junit4]   2> 669 T383 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080003, negotiated timeout = 20000
[junit4:junit4]   2> 670 T384 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6bf90542 name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 670 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 673 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 681 T362 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 685 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65252_e_li%2Fw
[junit4:junit4]   2> 686 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:65252_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:65252_e_li%2Fw
[junit4:junit4]   2> 688 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65252_e_li%2Fw
[junit4:junit4]   2> 694 T362 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 710 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 711 T362 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 717 T362 oasc.Overseer.start Overseer (id=89070042293469187-127.0.0.1:65252_e_li%2Fw-n_0000000000) starting
[junit4:junit4]   2> 718 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 720 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 728 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 730 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 736 T386 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 738 T362 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 750 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 758 T385 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 769 T387 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1
[junit4:junit4]   2> 770 T387 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 771 T387 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 771 T387 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 775 T387 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1/'
[junit4:junit4]   2> 777 T387 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1/lib/README' to classloader
[junit4:junit4]   2> 777 T387 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 833 T387 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 930 T387 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1032 T387 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1048 T387 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1763 T387 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1776 T387 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1781 T387 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1805 T387 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1813 T387 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1821 T387 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1824 T387 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1826 T387 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/
[junit4:junit4]   2> 1826 T387 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 1827 T387 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 1862 T387 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data forceNew:false
[junit4:junit4]   2> 1863 T387 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data
[junit4:junit4]   2> 1863 T387 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index/
[junit4:junit4]   2> 1864 T387 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1865 T387 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index forceNew:false
[junit4:junit4]   2> 1873 T387 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a7e9db9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1873 T387 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1874 T387 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index
[junit4:junit4]   2> 1874 T387 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1877 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1877 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1877 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1878 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1879 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1879 T387 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1879 T387 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1880 T387 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1881 T387 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1881 T387 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1882 T387 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1882 T387 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1883 T387 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1883 T387 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1884 T387 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1884 T387 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1885 T387 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1886 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1886 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1887 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1887 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1888 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1889 T387 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1889 T387 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1890 T387 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1891 T387 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1905 T387 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1913 T387 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1915 T387 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1919 T387 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data
[junit4:junit4]   2> 1922 T387 oass.SolrIndexSearcher.<init> Opening Searcher@66ab98b6 main
[junit4:junit4]   2> 1923 T387 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1923 T387 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1924 T387 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1955 T387 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2106 T388 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@66ab98b6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2111 T387 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 2111 T387 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2112 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2265 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2266 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65252_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65252/e_li/w"}
[junit4:junit4]   2> 2267 T385 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2301 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2302 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2327 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3127 T387 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 3127 T387 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65252/e_li/w collection:control_collection shard:shard1
[junit4:junit4]   2> 3143 T387 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 3163 T387 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 3185 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 3186 T387 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 3187 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3191 T387 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 3191 T387 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 3192 T387 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:65252/e_li/w/collection1/
[junit4:junit4]   2> 3192 T387 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 3193 T387 oasc.SyncStrategy.syncToMe http://127.0.0.1:65252/e_li/w/collection1/ has no replicas
[junit4:junit4]   2> 3193 T387 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:65252/e_li/w/collection1/
[junit4:junit4]   2> 3194 T387 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 3201 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3835 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3854 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3919 T387 oasc.ZkController.register We are http://127.0.0.1:65252/e_li/w/collection1/ and leader is http://127.0.0.1:65252/e_li/w/collection1/
[junit4:junit4]   2> 3920 T387 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65252/e_li/w
[junit4:junit4]   2> 3920 T387 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3920 T387 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3921 T387 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3925 T387 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3927 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3928 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3929 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3944 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3963 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@543ff9fb
[junit4:junit4]   2> 3964 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3965 T389 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3966 T389 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 3966 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65258
[junit4:junit4]   2> 3967 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65258
[junit4:junit4]   2> 3968 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080004 with negotiated timeout 10000 for client /140.211.11.196:65258
[junit4:junit4]   2> 3968 T389 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080004, negotiated timeout = 10000
[junit4:junit4]   2> 3969 T390 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@543ff9fb name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3969 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3972 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3979 T362 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4158 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4161 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65259
[junit4:junit4]   2> 4162 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4163 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4163 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095
[junit4:junit4]   2> 4164 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/solr.xml
[junit4:junit4]   2> 4165 T362 oasc.CoreContainer.<init> New CoreContainer 1473632271
[junit4:junit4]   2> 4165 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/'
[junit4:junit4]   2> 4166 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/'
[junit4:junit4]   2> 4225 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4226 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4227 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4227 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4228 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4228 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4229 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4230 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4230 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4231 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4258 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4284 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 4285 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4286 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@508041a6
[junit4:junit4]   2> 4287 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4288 T400 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4289 T400 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 4289 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65260
[junit4:junit4]   2> 4289 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65260
[junit4:junit4]   2> 4292 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080005 with negotiated timeout 20000 for client /140.211.11.196:65260
[junit4:junit4]   2> 4292 T400 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080005, negotiated timeout = 20000
[junit4:junit4]   2> 4292 T401 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@508041a6 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4293 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4295 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080005
[junit4:junit4]   2> 4304 T401 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4304 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080005 closed
[junit4:junit4]   2> 4304 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65260 which had sessionid 0x13c70bb47080005
[junit4:junit4]   2> 4305 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4320 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@16336f1a
[junit4:junit4]   2> 4322 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4322 T402 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4323 T402 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 4323 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65261
[junit4:junit4]   2> 4324 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65261
[junit4:junit4]   2> 4325 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080006 with negotiated timeout 20000 for client /140.211.11.196:65261
[junit4:junit4]   2> 4325 T402 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080006, negotiated timeout = 20000
[junit4:junit4]   2> 4326 T403 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16336f1a name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4326 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4328 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4330 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4337 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65259_e_li%2Fw
[junit4:junit4]   2> 4338 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:65259_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:65259_e_li%2Fw
[junit4:junit4]   2> 4340 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65259_e_li%2Fw
[junit4:junit4]   2> 4343 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4345 T390 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4348 T384 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4352 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4361 T404 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1
[junit4:junit4]   2> 4361 T404 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4362 T404 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4363 T404 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4365 T404 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1/'
[junit4:junit4]   2> 4366 T404 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1/lib/README' to classloader
[junit4:junit4]   2> 4367 T404 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4435 T404 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4530 T404 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4631 T404 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4646 T404 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5312 T404 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5320 T404 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5323 T404 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5339 T404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5343 T404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5347 T404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5348 T404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5349 T404 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/
[junit4:junit4]   2> 5349 T404 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 5350 T404 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 5351 T404 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1 forceNew:false
[junit4:junit4]   2> 5351 T404 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 5351 T404 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index/
[junit4:junit4]   2> 5352 T404 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5353 T404 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index forceNew:false
[junit4:junit4]   2> 5357 T404 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f148d7f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5357 T404 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5358 T404 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index
[junit4:junit4]   2> 5358 T404 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5360 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5360 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5360 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5361 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5361 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5361 T404 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5362 T404 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5362 T404 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5363 T404 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5363 T404 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5364 T404 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5364 T404 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5365 T404 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5365 T404 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5365 T404 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5366 T404 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5366 T404 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5367 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5368 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5368 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5369 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5370 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5370 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5370 T404 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5371 T404 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5371 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65252_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65252/e_li/w"}
[junit4:junit4]   2> 5372 T404 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5372 T404 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5381 T404 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5384 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5384 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5384 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5385 T404 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5387 T404 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5390 T404 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 5391 T404 oass.SolrIndexSearcher.<init> Opening Searcher@45d430ea main
[junit4:junit4]   2> 5392 T404 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5392 T404 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5393 T404 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5398 T404 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5444 T405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@45d430ea main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5449 T404 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5449 T404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6892 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6893 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65259_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65259/e_li/w"}
[junit4:junit4]   2> 6893 T385 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 6894 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6929 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6929 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6929 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7453 T404 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7453 T404 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65259/e_li/w collection:collection1 shard:shard1
[junit4:junit4]   2> 7454 T404 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7477 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7478 T404 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7479 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7489 T404 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7490 T404 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7490 T404 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:65259/e_li/w/collection1/
[junit4:junit4]   2> 7490 T404 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7491 T404 oasc.SyncStrategy.syncToMe http://127.0.0.1:65259/e_li/w/collection1/ has no replicas
[junit4:junit4]   2> 7491 T404 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:65259/e_li/w/collection1/
[junit4:junit4]   2> 7491 T404 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7498 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8436 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8484 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8484 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8484 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8526 T404 oasc.ZkController.register We are http://127.0.0.1:65259/e_li/w/collection1/ and leader is http://127.0.0.1:65259/e_li/w/collection1/
[junit4:junit4]   2> 8526 T404 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65259/e_li/w
[junit4:junit4]   2> 8526 T404 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8527 T404 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8527 T404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8530 T404 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8532 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8532 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8533 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8718 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8721 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65264
[junit4:junit4]   2> 8722 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8722 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8723 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665
[junit4:junit4]   2> 8724 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/solr.xml
[junit4:junit4]   2> 8724 T362 oasc.CoreContainer.<init> New CoreContainer 161107304
[junit4:junit4]   2> 8725 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/'
[junit4:junit4]   2> 8725 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/'
[junit4:junit4]   2> 8781 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 8782 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8782 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8783 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8783 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8784 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8784 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8785 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8785 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8786 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8808 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8828 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 8829 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8829 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3597892
[junit4:junit4]   2> 8830 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8831 T415 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8832 T415 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 8832 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65265
[junit4:junit4]   2> 8833 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65265
[junit4:junit4]   2> 8835 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080007 with negotiated timeout 20000 for client /140.211.11.196:65265
[junit4:junit4]   2> 8835 T415 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080007, negotiated timeout = 20000
[junit4:junit4]   2> 8835 T416 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3597892 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8836 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8837 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080007
[junit4:junit4]   2> 8846 T416 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8846 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65265 which had sessionid 0x13c70bb47080007
[junit4:junit4]   2> 8846 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080007 closed
[junit4:junit4]   2> 8847 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 8859 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6b680257
[junit4:junit4]   2> 8860 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8860 T417 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8861 T417 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 8861 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65266
[junit4:junit4]   2> 8862 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65266
[junit4:junit4]   2> 8863 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080008 with negotiated timeout 20000 for client /140.211.11.196:65266
[junit4:junit4]   2> 8863 T417 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080008, negotiated timeout = 20000
[junit4:junit4]   2> 8864 T418 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b680257 name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8864 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8865 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8867 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8875 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65264_e_li%2Fw
[junit4:junit4]   2> 8875 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:65264_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:65264_e_li%2Fw
[junit4:junit4]   2> 8877 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65264_e_li%2Fw
[junit4:junit4]   2> 8880 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8880 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8880 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8882 T384 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8883 T390 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8884 T403 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8885 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 8893 T419 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1
[junit4:junit4]   2> 8894 T419 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 8895 T419 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 8895 T419 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 8897 T419 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1/'
[junit4:junit4]   2> 8898 T419 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1/lib/README' to classloader
[junit4:junit4]   2> 8899 T419 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 8955 T419 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9028 T419 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9129 T419 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9139 T419 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9748 T419 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9756 T419 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9759 T419 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9776 T419 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9780 T419 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9784 T419 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9786 T419 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9787 T419 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/
[junit4:junit4]   2> 9788 T419 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 9788 T419 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 9789 T419 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2 forceNew:false
[junit4:junit4]   2> 9789 T419 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 9790 T419 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index/
[junit4:junit4]   2> 9790 T419 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9791 T419 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index forceNew:false
[junit4:junit4]   2> 9795 T419 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c66c0a2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9795 T419 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 9796 T419 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index
[junit4:junit4]   2> 9796 T419 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9798 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9798 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9799 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9799 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9800 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9800 T419 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9800 T419 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9801 T419 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9801 T419 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9802 T419 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9802 T419 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9803 T419 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9803 T419 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9803 T419 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9804 T419 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9804 T419 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9805 T419 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9805 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9806 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9806 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9807 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9807 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9808 T419 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9809 T419 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9809 T419 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9810 T419 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9818 T419 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9821 T419 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9823 T419 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 9824 T419 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 9826 T419 oass.SolrIndexSearcher.<init> Opening Searcher@52a63e02 main
[junit4:junit4]   2> 9827 T419 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9827 T419 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9827 T419 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9833 T419 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 9880 T420 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@52a63e02 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 9885 T419 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 9885 T419 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9997 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9998 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65259_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65259/e_li/w"}
[junit4:junit4]   2> 10004 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65264_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65264/e_li/w"}
[junit4:junit4]   2> 10004 T385 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 10004 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 10016 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10016 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10016 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10016 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10889 T419 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 10889 T419 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65264/e_li/w collection:collection1 shard:shard2
[junit4:junit4]   2> 10890 T419 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 10914 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:delete cxid:0x42 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 10915 T419 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 10916 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:create cxid:0x43 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10927 T419 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 10927 T419 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 10928 T419 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:65264/e_li/w/collection1/
[junit4:junit4]   2> 10928 T419 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 10928 T419 oasc.SyncStrategy.syncToMe http://127.0.0.1:65264/e_li/w/collection1/ has no replicas
[junit4:junit4]   2> 10929 T419 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:65264/e_li/w/collection1/
[junit4:junit4]   2> 10929 T419 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 10934 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:create cxid:0x4c zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11531 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11554 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11554 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11554 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11554 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11599 T419 oasc.ZkController.register We are http://127.0.0.1:65264/e_li/w/collection1/ and leader is http://127.0.0.1:65264/e_li/w/collection1/
[junit4:junit4]   2> 11599 T419 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65264/e_li/w
[junit4:junit4]   2> 11599 T419 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 11600 T419 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 11600 T419 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11602 T419 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11603 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11604 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11605 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11793 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 11797 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:33672
[junit4:junit4]   2> 11797 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11798 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11799 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737
[junit4:junit4]   2> 11799 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/solr.xml
[junit4:junit4]   2> 11800 T362 oasc.CoreContainer.<init> New CoreContainer 2124424930
[junit4:junit4]   2> 11800 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/'
[junit4:junit4]   2> 11801 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/'
[junit4:junit4]   2> 11856 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 11857 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11857 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 11858 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11858 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11859 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11859 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11860 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11860 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11861 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 11885 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11905 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 11906 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 11906 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@59f382ac
[junit4:junit4]   2> 11908 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11908 T430 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11909 T430 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 11910 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22663
[junit4:junit4]   2> 11910 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22663
[junit4:junit4]   2> 11923 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080009 with negotiated timeout 20000 for client /140.211.11.196:22663
[junit4:junit4]   2> 11923 T430 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080009, negotiated timeout = 20000
[junit4:junit4]   2> 11924 T431 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59f382ac name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11924 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11927 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080009
[junit4:junit4]   2> 11946 T431 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 11946 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22663 which had sessionid 0x13c70bb47080009
[junit4:junit4]   2> 11946 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080009 closed
[junit4:junit4]   2> 11947 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 11962 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1afddef2
[junit4:junit4]   2> 11963 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11963 T432 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11964 T432 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 11964 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:44735
[junit4:junit4]   2> 11965 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:44735
[junit4:junit4]   2> 11975 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb4708000a with negotiated timeout 20000 for client /140.211.11.196:44735
[junit4:junit4]   2> 11976 T432 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb4708000a, negotiated timeout = 20000
[junit4:junit4]   2> 11976 T433 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1afddef2 name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11977 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11979 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11992 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11999 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33672_e_li%2Fw
[junit4:junit4]   2> 12000 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:33672_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:33672_e_li%2Fw
[junit4:junit4]   2> 12006 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33672_e_li%2Fw
[junit4:junit4]   2> 12010 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12010 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12010 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12010 T418 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12011 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12018 T403 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12018 T384 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12018 T390 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12020 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12029 T434 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1
[junit4:junit4]   2> 12029 T434 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12030 T434 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12030 T434 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12032 T434 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1/'
[junit4:junit4]   2> 12033 T434 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1/lib/README' to classloader
[junit4:junit4]   2> 12034 T434 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12089 T434 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12187 T434 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12288 T434 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12298 T434 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12931 T434 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 12944 T434 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 12959 T434 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12980 T434 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12986 T434 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12991 T434 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12993 T434 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12995 T434 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/
[junit4:junit4]   2> 12995 T434 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 12996 T434 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 12997 T434 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3 forceNew:false
[junit4:junit4]   2> 12997 T434 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 12998 T434 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index/
[junit4:junit4]   2> 12998 T434 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12999 T434 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index forceNew:false
[junit4:junit4]   2> 13042 T434 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@52dd2f52),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13043 T434 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13043 T434 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index
[junit4:junit4]   2> 13044 T434 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13047 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13047 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13047 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13048 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13049 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13049 T434 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13050 T434 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13050 T434 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13051 T434 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13052 T434 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13053 T434 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13053 T434 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13054 T434 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13054 T434 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13055 T434 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13056 T434 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13056 T434 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13057 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13058 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13059 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13060 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13060 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13061 T434 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13062 T434 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13063 T434 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13063 T434 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13068 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13069 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65264_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65264/e_li/w"}
[junit4:junit4]   2> 13076 T434 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13081 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T434 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13085 T434 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13088 T434 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 13090 T434 oass.SolrIndexSearcher.<init> Opening Searcher@23ac1194 main
[junit4:junit4]   2> 13091 T434 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13091 T434 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13092 T434 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13099 T434 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13153 T435 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23ac1194 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13161 T434 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13161 T434 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14587 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14589 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:33672_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:33672/e_li/w"}
[junit4:junit4]   2> 14589 T385 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14589 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 14605 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14606 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14606 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14606 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14606 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15165 T434 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15165 T434 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33672/e_li/w collection:collection1 shard:shard1
[junit4:junit4]   2> 15188 T434 oasc.ZkController.register We are http://127.0.0.1:33672/e_li/w/collection1/ and leader is http://127.0.0.1:65259/e_li/w/collection1/
[junit4:junit4]   2> 15188 T434 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33672/e_li/w
[junit4:junit4]   2> 15189 T434 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15189 T434 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C33 name=collection1 org.apache.solr.core.SolrCore@717a69a8 url=http://127.0.0.1:33672/e_li/w/collection1 node=127.0.0.1:33672_e_li%2Fw C33_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:33672_e_li%2Fw, base_url=http://127.0.0.1:33672/e_li/w}
[junit4:junit4]   2> 15201 T436 C33 P33672 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15202 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15203 T436 C33 P33672 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15203 T436 C33 P33672 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15219 T434 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15221 T436 C33 P33672 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15222 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 15222 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15223 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15446 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15449 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10176
[junit4:junit4]   2> 15450 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15450 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15450 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363
[junit4:junit4]   2> 15451 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/solr.xml
[junit4:junit4]   2> 15451 T362 oasc.CoreContainer.<init> New CoreContainer 2028850196
[junit4:junit4]   2> 15452 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/'
[junit4:junit4]   2> 15452 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/'
[junit4:junit4]   2> 15502 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 15502 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15503 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15503 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15503 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15504 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15504 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15504 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15505 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15505 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15522 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15537 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 15538 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15538 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4d39f2f6
[junit4:junit4]   2> 15539 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15539 T446 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15540 T446 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 15540 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18886
[junit4:junit4]   2> 15541 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18886
[junit4:junit4]   2> 15550 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb4708000b with negotiated timeout 20000 for client /140.211.11.196:18886
[junit4:junit4]   2> 15550 T446 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb4708000b, negotiated timeout = 20000
[junit4:junit4]   2> 15551 T447 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d39f2f6 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15551 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15552 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb4708000b
[junit4:junit4]   2> 15556 T447 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15556 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18886 which had sessionid 0x13c70bb4708000b
[junit4:junit4]   2> 15556 T362 oaz.ZooKeeper.close Session: 0x13c70bb4708000b closed
[junit4:junit4]   2> 15557 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 15565 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5173890f
[junit4:junit4]   2> 15565 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15566 T448 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15566 T448 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 15566 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:29970
[junit4:junit4]   2> 15567 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:29970
[junit4:junit4]   2> 15575 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb4708000c with negotiated timeout 20000 for client /140.211.11.196:29970
[junit4:junit4]   2> 15575 T448 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb4708000c, negotiated timeout = 20000
[junit4:junit4]   2> 15576 T449 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5173890f name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15576 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15577 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000c type:create cxid:0x1 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15583 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000c type:create cxid:0x2 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15594 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10176_e_li%2Fw
[junit4:junit4]   2> 15595 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000c type:delete cxid:0x4 zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:10176_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:10176_e_li%2Fw
[junit4:junit4]   2> 15596 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10176_e_li%2Fw
[junit4:junit4]   2> 15604 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15604 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15604 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15604 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15605 T418 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15605 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15607 T384 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15607 T403 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15608 T433 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15607 T390 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15609 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15616 T450 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1
[junit4:junit4]   2> 15616 T450 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15617 T450 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15617 T450 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15619 T450 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1/'
[junit4:junit4]   2> 15621 T450 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1/lib/README' to classloader
[junit4:junit4]   2> 15621 T450 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15681 T450 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 15759 T450 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 15861 T450 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 15869 T450 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 16112 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16114 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:33672_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:33672/e_li/w"}
[junit4:junit4]   2> 16224 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16225 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16224 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16224 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16224 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16225 T449 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16544 T450 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16557 T450 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16561 T450 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16582 T450 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16589 T450 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16594 T450 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16596 T450 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16598 T450 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/
[junit4:junit4]   2> 16598 T450 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 16599 T450 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 16600 T450 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4 forceNew:false
[junit4:junit4]   2> 16601 T450 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 16601 T450 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index/
[junit4:junit4]   2> 16602 T450 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16603 T450 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index forceNew:false
[junit4:junit4]   2> 16612 T450 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@84efce4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16613 T450 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 16614 T450 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index
[junit4:junit4]   2> 16614 T450 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16617 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16618 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16618 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16618 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16619 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16620 T450 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16620 T450 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16621 T450 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16622 T450 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16623 T450 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16624 T450 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16624 T450 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16625 T450 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16625 T450 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16626 T450 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16627 T450 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16628 T450 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16628 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16629 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16630 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16631 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16631 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16632 T450 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16633 T450 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16634 T450 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16635 T450 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16651 T450 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16657 T450 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16659 T450 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16662 T450 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 16664 T450 oass.SolrIndexSearcher.<init> Opening Searcher@27fa1b48 main
[junit4:junit4]   2> 16665 T450 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16665 T450 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16666 T450 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16675 T450 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 16787 T451 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@27fa1b48 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 16795 T450 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 16795 T450 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17745 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17747 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10176_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10176/e_li/w"}
[junit4:junit4]   2> 17747 T385 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 17747 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 17762 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T449 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17799 T450 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 17799 T450 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10176/e_li/w collection:collection1 shard:shard2
[junit4:junit4]   2> 17804 T450 oasc.ZkController.register We are http://127.0.0.1:10176/e_li/w/collection1/ and leader is http://127.0.0.1:65264/e_li/w/collection1/
[junit4:junit4]   2> 17804 T450 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10176/e_li/w
[junit4:junit4]   2> 17805 T450 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 17805 T450 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C34 name=collection1 org.apache.solr.core.SolrCore@6f8cf13b url=http://127.0.0.1:10176/e_li/w/collection1 node=127.0.0.1:10176_e_li%2Fw C34_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:10176_e_li%2Fw, base_url=http://127.0.0.1:10176/e_li/w}
[junit4:junit4]   2> 17806 T452 C34 P10176 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 17806 T450 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17807 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 17807 T452 C34 P10176 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 17807 T452 C34 P10176 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17808 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 17809 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 17809 T452 C34 P10176 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17809 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17829 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17832 T362 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 17833 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C33_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:33672_e_li%2Fw, base_url=http://127.0.0.1:33672/e_li/w}
[junit4:junit4]   2> 18278 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:65259/e_li/w/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18289 T436 C33 P33672 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 18309 T436 C33 P33672 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:33672/e_li/w START replicas=[http://127.0.0.1:65259/e_li/w/collection1/] nUpdates=100
[junit4:junit4]   2> 18322 T436 C33 P33672 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18323 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18323 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18323 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18324 T436 C33 P33672 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18324 T436 C33 P33672 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:65259/e_li/w/collection1/. core=collection1
[junit4:junit4]   2> 18324 T436 C33 P33672 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C35 name=collection1 org.apache.solr.core.SolrCore@d3430e url=http://127.0.0.1:65259/e_li/w/collection1 node=127.0.0.1:65259_e_li%2Fw C35_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:65259_e_li%2Fw, base_url=http://127.0.0.1:65259/e_li/w, leader=true}
[junit4:junit4]   2> 18344 T394 C35 P65259 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18348 T395 C35 P65259 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18349 T395 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 18353 T395 C35 P65259 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f148d7f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18354 T395 C35 P65259 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18362 T395 C35 P65259 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f148d7f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f148d7f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18362 T395 C35 P65259 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18364 T395 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 18364 T395 C35 P65259 oass.SolrIndexSearcher.<init> Opening Searcher@20ec3516 realtime
[junit4:junit4]   2> 18365 T395 C35 P65259 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18365 T395 C35 P65259 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4]   2> 18426 T436 C33 P33672 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18427 T436 C33 P33672 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 18455 T436 C33 P33672 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18458 T396 C35 P65259 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18459 T396 C35 P65259 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18460 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18460 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18460 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18464 T396 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 18465 T396 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index
[junit4:junit4]   2> 18465 T396 C35 P65259 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18466 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18468 T436 C33 P33672 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 forceNew:false
[junit4:junit4]   2> 18469 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18469 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5) fullCopy=true
[junit4:junit4]   2> 18485 T396 C35 P65259 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=6 
[junit4:junit4]   2> 18492 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index
[junit4:junit4]   2> 18492 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18495 T436 C33 P33672 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130125100325583
[junit4:junit4]   2> 18497 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18498 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18499 T436 C33 P33672 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583
[junit4:junit4]   2> 18503 T436 C33 P33672 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18503 T436 C33 P33672 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18505 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18510 T436 C33 P33672 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18511 T436 C33 P33672 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18511 T436 C33 P33672 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18512 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583
[junit4:junit4]   2> 18513 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18516 T436 C33 P33672 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18517 T436 C33 P33672 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18517 T436 C33 P33672 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18518 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18518 T436 C33 P33672 oass.SolrIndexSearcher.<init> Opening Searcher@5ac62fa9 main
[junit4:junit4]   2> 18522 T435 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5ac62fa9 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18523 T435 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index
[junit4:junit4]   2> 18523 T436 C33 P33672 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18537 T436 C33 P33672 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18538 T436 C33 P33672 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18539 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18539 T436 C33 P33672 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18540 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583
[junit4:junit4]   2> 18540 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index
[junit4:junit4]   2> 18540 T436 C33 P33672 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18541 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18541 T436 C33 P33672 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18541 T436 C33 P33672 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18543 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 18836 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19269 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19271 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10176_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10176/e_li/w"}
[junit4:junit4]   2> 19304 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:33672_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:33672/e_li/w"}
[junit4:junit4]   2> 19329 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T449 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19838 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20842 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C34_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:10176_e_li%2Fw, base_url=http://127.0.0.1:10176/e_li/w}
[junit4:junit4]   2> 21833 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:65264/e_li/w/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 21834 T452 C34 P10176 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:10176/e_li/w START replicas=[http://127.0.0.1:65264/e_li/w/collection1/] nUpdates=100
[junit4:junit4]   2> 21834 T452 C34 P10176 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 21835 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 21836 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 21836 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 21836 T452 C34 P10176 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 21837 T452 C34 P10176 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:65264/e_li/w/collection1/. core=collection1
[junit4:junit4]   2> 21837 T452 C34 P10176 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 21845 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C36 name=collection1 org.apache.solr.core.SolrCore@5373a8a3 url=http://127.0.0.1:65264/e_li/w/collection1 node=127.0.0.1:65264_e_li%2Fw C36_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:65264_e_li%2Fw, base_url=http://127.0.0.1:65264/e_li/w, leader=true}
[junit4:junit4]   2> 21850 T409 C36 P65264 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 21857 T410 C36 P65264 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 21859 T410 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 21864 T410 C36 P65264 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c66c0a2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21865 T410 C36 P65264 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21891 T410 C36 P65264 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c66c0a2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c66c0a2),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21892 T410 C36 P65264 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21893 T410 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 21894 T410 C36 P65264 oass.SolrIndexSearcher.<init> Opening Searcher@47402d88 realtime
[junit4:junit4]   2> 21894 T410 C36 P65264 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21895 T410 C36 P65264 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 38
[junit4:junit4]   2> 21896 T452 C34 P10176 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21897 T452 C34 P10176 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 21900 T411 C36 P65264 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21901 T411 C36 P65264 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 21902 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 21902 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 21902 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 21906 T411 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 21906 T411 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index
[junit4:junit4]   2> 21907 T411 C36 P65264 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 21908 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 21909 T452 C34 P10176 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 forceNew:false
[junit4:junit4]   2> 21910 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21911 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63) fullCopy=true
[junit4:junit4]   2> 21914 T411 C36 P65264 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 21916 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index
[junit4:junit4]   2> 21917 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 21935 T452 C34 P10176 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130125100329025
[junit4:junit4]   2> 21936 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21937 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21937 T452 C34 P10176 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025
[junit4:junit4]   2> 21941 T452 C34 P10176 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21942 T452 C34 P10176 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21944 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21944 T452 C34 P10176 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 21945 T452 C34 P10176 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 21945 T452 C34 P10176 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 21946 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025
[junit4:junit4]   2> 21947 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21951 T452 C34 P10176 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21951 T452 C34 P10176 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21952 T452 C34 P10176 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 21952 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21953 T452 C34 P10176 oass.SolrIndexSearcher.<init> Opening Searcher@4bf9418e main
[junit4:junit4]   2> 21959 T451 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4bf9418e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 21960 T451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index
[junit4:junit4]   2> 21960 T452 C34 P10176 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 21992 T452 C34 P10176 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 21992 T452 C34 P10176 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 21993 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21994 T452 C34 P10176 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21994 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025
[junit4:junit4]   2> 21994 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index
[junit4:junit4]   2> 21995 T452 C34 P10176 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 21995 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 21995 T452 C34 P10176 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 21995 T452 C34 P10176 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22020 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 22347 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22349 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10176_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10176/e_li/w"}
[junit4:junit4]   2> 22359 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22360 T449 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22360 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22360 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22359 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22360 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22849 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22850 T362 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C37 name=collection1 org.apache.solr.core.SolrCore@36ae2b0 url=http://127.0.0.1:65252/e_li/w/collection1 node=127.0.0.1:65252_e_li%2Fw C37_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:65252_e_li%2Fw, base_url=http://127.0.0.1:65252/e_li/w, leader=true}
[junit4:junit4]   2> 22863 T374 C37 P65252 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data
[junit4:junit4]   2> 22867 T374 C37 P65252 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a7e9db9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 22868 T374 C37 P65252 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 22869 T374 C37 P65252 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data
[junit4:junit4]   2> 22870 T374 C37 P65252 oass.SolrIndexSearcher.<init> Opening Searcher@46ab0975 realtime
[junit4:junit4]   2> 22870 T374 C37 P65252 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425120700638691328)} 0 9
[junit4:junit4]   2> 22888 T397 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 22893 T412 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2>  C33_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:33672_e_li%2Fw, base_url=http://127.0.0.1:33672/e_li/w}
[junit4:junit4]   2> 22901 T423 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 22902 T423 C33 P33672 /update {update.distrib=FROMLEADER&_version_=-1425120700663857152&update.from=http://127.0.0.1:65259/e_li/w/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425120700663857152)} 0 3
[junit4:junit4]   2>  C34_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:10176_e_li%2Fw, base_url=http://127.0.0.1:10176/e_li/w}
[junit4:junit4]   2> 22906 T439 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 22907 T439 C34 P10176 /update {update.distrib=FROMLEADER&_version_=-1425120700669100032&update.from=http://127.0.0.1:65264/e_li/w/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425120700669100032)} 0 3
[junit4:junit4]   2> 22908 T412 C36 P65264 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425120700669100032)} 0 17
[junit4:junit4]   2> 22909 T397 C35 P65259 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425120700663857152)} 0 23
[junit4:junit4]   2> 22918 T374 C37 P65252 /update {wt=javabin&version=2} {add=[1 (1425120700693217280)]} 0 5
[junit4:junit4]   2> 22939 T423 C33 P33672 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425120700707897344)]} 0 5
[junit4:junit4]   2> 22940 T398 C35 P65259 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1425120700707897344)]} 0 12
[junit4:junit4]   2> 22941 T413 C36 P65264 /update {wt=javabin&version=2} {add=[1]} 0 21
[junit4:junit4]   2> 22947 T374 C37 P65252 /update {wt=javabin&version=2} {add=[2 (1425120700725723136)]} 0 3
[junit4:junit4]   2> 22961 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425120700732014592)]} 0 4
[junit4:junit4]   2> 22962 T413 C36 P65264 /update {wt=javabin&version=2} {add=[2 (1425120700732014592)]} 0 12
[junit4:junit4]   2> 22968 T374 C37 P65252 /update {wt=javabin&version=2} {add=[3 (1425120700747743232)]} 0 2
[junit4:junit4]   2> 22981 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425120700757180416)]} 0 2
[junit4:junit4]   2> 22982 T412 C36 P65264 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1425120700757180416)]} 0 8
[junit4:junit4]   2> 22983 T397 C35 P65259 /update {wt=javabin&version=2} {add=[3]} 0 12
[junit4:junit4]   2> 22989 T374 C37 P65252 /update {wt=javabin&version=2} {add=[4 (1425120700769763328)]} 0 3
[junit4:junit4]   2> 23004 T423 C33 P33672 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425120700779200512)]} 0 3
[junit4:junit4]   2> 23005 T398 C35 P65259 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1425120700779200512)]} 0 9
[junit4:junit4]   2> 23006 T413 C36 P65264 /update {wt=javabin&version=2} {add=[4]} 0 14
[junit4:junit4]   2> 23012 T374 C37 P65252 /update {wt=javabin&version=2} {add=[5 (1425120700793880576)]} 0 3
[junit4:junit4]   2> 23026 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425120700802269184)]} 0 3
[junit4:junit4]   2> 23027 T412 C36 P65264 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1425120700802269184)]} 0 10
[junit4:junit4]   2> 23028 T397 C35 P65259 /update {wt=javabin&version=2} {add=[5]} 0 13
[junit4:junit4]   2> 23034 T374 C37 P65252 /update {wt=javabin&version=2} {add=[6 (1425120700816949248)]} 0 3
[junit4:junit4]   2> 23050 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425120700827435008)]} 0 3
[junit4:junit4]   2> 23051 T412 C36 P65264 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1425120700827435008)]} 0 10
[junit4:junit4]   2> 23052 T397 C35 P65259 /update {wt=javabin&version=2} {add=[6]} 0 15
[junit4:junit4]   2> 23060 T374 C37 P65252 /update {wt=javabin&version=2} {add=[7 (1425120700843163648)]} 0 4
[junit4:junit4]   2> 23072 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425120700851552256)]} 0 3
[junit4:junit4]   2> 23073 T413 C36 P65264 /update {wt=javabin&version=2} {add=[7 (1425120700851552256)]} 0 9
[junit4:junit4]   2> 23080 T374 C37 P65252 /update {wt=javabin&version=2} {add=[8 (1425120700865183744)]} 0 3
[junit4:junit4]   2> 23094 T423 C33 P33672 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425120700871475200)]} 0 4
[junit4:junit4]   2> 23095 T397 C35 P65259 /update {wt=javabin&version=2} {add=[8 (1425120700871475200)]} 0 11
[junit4:junit4]   2> 23102 T374 C37 P65252 /update {wt=javabin&version=2} {add=[9 (1425120700888252416)]} 0 3
[junit4:junit4]   2> 23113 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425120700894543872)]} 0 3
[junit4:junit4]   2> 23114 T413 C36 P65264 /update {wt=javabin&version=2} {add=[9 (1425120700894543872)]} 0 9
[junit4:junit4]   2> 23138 T374 C37 P65252 /update {wt=javabin&version=2} {add=[10 (1425120700908175360)]} 0 20
[junit4:junit4]   2> 23154 T423 C33 P33672 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425120700936486912)]} 0 3
[junit4:junit4]   2> 23155 T398 C35 P65259 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1425120700936486912)]} 0 10
[junit4:junit4]   2> 23156 T413 C36 P

[...truncated too long message...]

ger-9, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  35) Thread[id=507, name=searcherExecutor-287-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  36) Thread[id=542, name=searcherExecutor-309-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  37) Thread[id=372, name=HashSessionScavenger-6, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  38) Thread[id=405, name=searcherExecutor-212-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  39) Thread[id=388, name=searcherExecutor-202-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  40) Thread[id=543, name=searcherExecutor-310-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  41) Thread[id=389, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[AD8248BC50D3AF8D]-SendThread(localhost:65249), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:82)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  42) Thread[id=578, name=searcherExecutor-339-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=en_SG, timezone=Africa/Gaborone
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=45,free=166577424,total=263192576
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SpellPossibilityIteratorTest, SystemInfoHandlerTest, TestQueryTypes, CopyFieldTest, TestLMJelinekMercerSimilarityFactory, CacheHeaderTest, TestElisionMultitermQuery, WordBreakSolrSpellCheckerTest, IndexBasedSpellCheckerTest, TestConfig, SpellCheckCollatorTest, TestQuerySenderNoQuery, TestAtomicUpdateErrorCases, MoreLikeThisHandlerTest, TermsComponentTest, LoggingHandlerTest, TestDefaultSimilarityFactory, TestLFUCache, SolrCmdDistributorTest, FileBasedSpellCheckerTest, AutoCommitTest, FieldAnalysisRequestHandlerTest, CSVRequestHandlerTest, QueryElevationComponentTest, UpdateRequestProcessorFactoryTest, TestXIncludeConfig, ZkControllerTest, TestLuceneMatchVersion, DateFieldTest, SampleTest, FastVectorHighlighterTest, TestWordDelimiterFilterFactory, TestPerFieldSimilarity, DateMathParserTest, PreAnalyzedFieldTest, ChaosMonkeyNothingIsSafeTest, JSONWriterTest, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=AD8248BC50D3AF8D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_SG -Dtests.timezone=Africa/Gaborone -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([AD8248BC50D3AF8D]:0)
[junit4:junit4] Completed on J0 in 7223.98s, 1 test, 2 errors <<< FAILURES!

[...truncated 74 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 247 suites, 1032 tests, 1 suite-level error, 1 error, 66 ignored (5 assumptions)

Total time: 141 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3677 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3677/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.ShardRoutingTest.testDistribSearch

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:63087/e_vr/g/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:63087/e_vr/g/collection1
	at __randomizedtesting.SeedInfo.seed([73DB5873A286C061:F23DD66BD5D9A05D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
	at org.apache.solr.cloud.ShardRoutingTest.doTestNumRequests(ShardRoutingTest.java:224)
	at org.apache.solr.cloud.ShardRoutingTest.doTest(ShardRoutingTest.java:120)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
	... 47 more


FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([73DB5873A286C061]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([73DB5873A286C061]:0)




Build Log:
[...truncated 9323 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ShardRoutingTest
[junit4:junit4]   2> 0 T3128 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /e_vr/g
[junit4:junit4]   2> 5 T3128 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-ShardRoutingTest-1359127485054
[junit4:junit4]   2> 6 T3128 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T3129 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T3129 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T3129 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T3129 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T3129 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T3129 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T3128 oasc.ZkTestServer.run start zk server on port:63079
[junit4:junit4]   2> 108 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1687cd23
[junit4:junit4]   2> 109 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T3134 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T3134 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 110 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63080
[junit4:junit4]   2> 111 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63080
[junit4:junit4]   2> 111 T3132 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 113 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850000 with negotiated timeout 10000 for client /140.211.11.196:63080
[junit4:junit4]   2> 113 T3134 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850000, negotiated timeout = 10000
[junit4:junit4]   2> 114 T3135 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1687cd23 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 114 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 115 T3128 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 117 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850000
[junit4:junit4]   2> 129 T3135 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 130 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63080 which had sessionid 0x13c724f9a850000
[junit4:junit4]   2> 129 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850000 closed
[junit4:junit4]   2> 130 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2aaaab79
[junit4:junit4]   2> 131 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 131 T3136 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 132 T3136 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 132 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63081
[junit4:junit4]   2> 132 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63081
[junit4:junit4]   2> 133 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850001 with negotiated timeout 10000 for client /140.211.11.196:63081
[junit4:junit4]   2> 133 T3136 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850001, negotiated timeout = 10000
[junit4:junit4]   2> 134 T3137 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2aaaab79 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 134 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 135 T3128 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 143 T3128 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 146 T3128 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 148 T3128 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 151 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 152 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 167 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 167 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 274 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 275 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 278 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 278 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 281 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 282 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 285 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 285 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 288 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 289 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 296 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 297 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 300 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 301 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 304 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850001
[junit4:junit4]   2> 305 T3137 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 305 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63081 which had sessionid 0x13c724f9a850001
[junit4:junit4]   2> 305 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850001 closed
[junit4:junit4]   2> 474 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 479 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63082
[junit4:junit4]   2> 480 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 481 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 481 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354
[junit4:junit4]   2> 482 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/solr.xml
[junit4:junit4]   2> 482 T3128 oasc.CoreContainer.<init> New CoreContainer 1413403077
[junit4:junit4]   2> 483 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/'
[junit4:junit4]   2> 483 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/'
[junit4:junit4]   2> 540 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 541 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 542 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 542 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 543 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 543 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 544 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 544 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 545 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 545 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 567 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 586 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 587 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 588 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@a00f995
[junit4:junit4]   2> 589 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 589 T3147 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 590 T3147 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 590 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63083
[junit4:junit4]   2> 591 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63083
[junit4:junit4]   2> 593 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850002 with negotiated timeout 20000 for client /140.211.11.196:63083
[junit4:junit4]   2> 593 T3147 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850002, negotiated timeout = 20000
[junit4:junit4]   2> 594 T3148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a00f995 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 594 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 595 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850002
[junit4:junit4]   2> 596 T3148 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 596 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63083 which had sessionid 0x13c724f9a850002
[junit4:junit4]   2> 596 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850002 closed
[junit4:junit4]   2> 597 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 608 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@35ff1f6f
[junit4:junit4]   2> 609 T3149 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 609 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 610 T3149 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 610 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63084
[junit4:junit4]   2> 610 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63084
[junit4:junit4]   2> 611 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850003 with negotiated timeout 20000 for client /140.211.11.196:63084
[junit4:junit4]   2> 612 T3149 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850003, negotiated timeout = 20000
[junit4:junit4]   2> 612 T3150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35ff1f6f name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 612 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 614 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 623 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 625 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63082_e_vr%2Fg
[junit4:junit4]   2> 626 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:63082_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:63082_e_vr%2Fg
[junit4:junit4]   2> 627 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63082_e_vr%2Fg
[junit4:junit4]   2> 636 T3128 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 649 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 650 T3128 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 657 T3128 oasc.Overseer.start Overseer (id=89071778860957699-127.0.0.1:63082_e_vr%2Fg-n_0000000000) starting
[junit4:junit4]   2> 657 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 659 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 660 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 662 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 673 T3152 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 673 T3128 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 676 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 678 T3151 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 683 T3153 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1
[junit4:junit4]   2> 683 T3153 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 684 T3153 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 684 T3153 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 685 T3153 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1/'
[junit4:junit4]   2> 686 T3153 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 687 T3153 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1/lib/README' to classloader
[junit4:junit4]   2> 742 T3153 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 813 T3153 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 913 T3153 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 919 T3153 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1501 T3153 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1505 T3153 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1522 T3153 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/
[junit4:junit4]   2> 1523 T3153 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 1523 T3153 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1525 T3153 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data forceNew:false
[junit4:junit4]   2> 1525 T3153 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data
[junit4:junit4]   2> 1526 T3153 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index/
[junit4:junit4]   2> 1526 T3153 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1527 T3153 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index forceNew:false
[junit4:junit4]   2> 1534 T3153 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@23d0ecc3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1534 T3153 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1535 T3153 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index
[junit4:junit4]   2> 1536 T3153 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1538 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1538 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1538 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1539 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1539 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1540 T3153 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1540 T3153 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1541 T3153 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1541 T3153 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1542 T3153 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1542 T3153 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1543 T3153 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1543 T3153 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1543 T3153 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1544 T3153 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1544 T3153 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1545 T3153 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1545 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1546 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1546 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1547 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1547 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1548 T3153 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1548 T3153 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1549 T3153 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1549 T3153 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1562 T3153 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1567 T3153 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1568 T3153 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1570 T3153 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data
[junit4:junit4]   2> 1572 T3153 oass.SolrIndexSearcher.<init> Opening Searcher@55fb709c main
[junit4:junit4]   2> 1573 T3153 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1574 T3153 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1574 T3153 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1591 T3153 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1647 T3154 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@55fb709c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1652 T3153 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1652 T3153 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1653 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x5c zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2181 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2181 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63082_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63082/e_vr/g"}
[junit4:junit4]   2> 2182 T3151 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2182 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2182 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x64 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2201 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2666 T3153 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2666 T3153 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63082/e_vr/g collection:control_collection shard:shard1
[junit4:junit4]   2> 2667 T3153 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2690 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:delete cxid:0x7e zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2691 T3153 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2691 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x7f zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2703 T3153 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2703 T3153 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2703 T3153 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63082/e_vr/g/collection1/
[junit4:junit4]   2> 2704 T3153 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2704 T3153 oasc.SyncStrategy.syncToMe http://127.0.0.1:63082/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 2704 T3153 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63082/e_vr/g/collection1/
[junit4:junit4]   2> 2705 T3153 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2714 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x89 zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3705 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3757 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3787 T3153 oasc.ZkController.register We are http://127.0.0.1:63082/e_vr/g/collection1/ and leader is http://127.0.0.1:63082/e_vr/g/collection1/
[junit4:junit4]   2> 3787 T3153 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63082/e_vr/g
[junit4:junit4]   2> 3787 T3153 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3787 T3153 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3788 T3153 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3789 T3153 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3790 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3791 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3791 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3805 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3814 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7319a2de
[junit4:junit4]   2> 3815 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3815 T3155 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3815 T3155 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 3816 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63086
[junit4:junit4]   2> 3816 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63086
[junit4:junit4]   2> 3817 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850004 with negotiated timeout 10000 for client /140.211.11.196:63086
[junit4:junit4]   2> 3817 T3155 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850004, negotiated timeout = 10000
[junit4:junit4]   2> 3818 T3156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7319a2de name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3818 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3819 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3820 T3128 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3986 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3990 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63087
[junit4:junit4]   2> 3991 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3992 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3992 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869
[junit4:junit4]   2> 3993 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/solr.xml
[junit4:junit4]   2> 3993 T3128 oasc.CoreContainer.<init> New CoreContainer 1255630714
[junit4:junit4]   2> 3994 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/'
[junit4:junit4]   2> 3995 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/'
[junit4:junit4]   2> 4052 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4053 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4053 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4054 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4054 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4055 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4055 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4056 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4056 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4057 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4076 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4096 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 4096 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4097 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@63f5739
[junit4:junit4]   2> 4098 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4098 T3166 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4099 T3166 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 4099 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63088
[junit4:junit4]   2> 4100 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63088
[junit4:junit4]   2> 4108 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850005 with negotiated timeout 20000 for client /140.211.11.196:63088
[junit4:junit4]   2> 4108 T3166 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850005, negotiated timeout = 20000
[junit4:junit4]   2> 4108 T3167 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63f5739 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4109 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4110 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850005
[junit4:junit4]   2> 4111 T3167 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4111 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63088 which had sessionid 0x13c724f9a850005
[junit4:junit4]   2> 4111 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850005 closed
[junit4:junit4]   2> 4112 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4121 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@43c50499
[junit4:junit4]   2> 4122 T3168 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4122 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4123 T3168 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 4123 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63089
[junit4:junit4]   2> 4123 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63089
[junit4:junit4]   2> 4125 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850006 with negotiated timeout 20000 for client /140.211.11.196:63089
[junit4:junit4]   2> 4125 T3168 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850006, negotiated timeout = 20000
[junit4:junit4]   2> 4125 T3169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43c50499 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4125 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4126 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4133 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4141 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63087_e_vr%2Fg
[junit4:junit4]   2> 4141 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:63087_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:63087_e_vr%2Fg
[junit4:junit4]   2> 4143 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63087_e_vr%2Fg
[junit4:junit4]   2> 4145 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4146 T3156 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4147 T3150 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4153 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4160 T3170 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1
[junit4:junit4]   2> 4160 T3170 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4161 T3170 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4161 T3170 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4162 T3170 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1/'
[junit4:junit4]   2> 4164 T3170 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4164 T3170 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1/lib/README' to classloader
[junit4:junit4]   2> 4219 T3170 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4290 T3170 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4391 T3170 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4397 T3170 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4978 T3170 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4983 T3170 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5001 T3170 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/
[junit4:junit4]   2> 5002 T3170 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 5002 T3170 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 5003 T3170 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1 forceNew:false
[junit4:junit4]   2> 5004 T3170 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1
[junit4:junit4]   2> 5004 T3170 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index/
[junit4:junit4]   2> 5005 T3170 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5006 T3170 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index forceNew:false
[junit4:junit4]   2> 5012 T3170 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b5485d6; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5012 T3170 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5013 T3170 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index
[junit4:junit4]   2> 5014 T3170 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5016 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5017 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5017 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5017 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5018 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5018 T3170 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5019 T3170 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5019 T3170 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5020 T3170 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5021 T3170 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5021 T3170 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5022 T3170 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5022 T3170 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5023 T3170 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5023 T3170 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5024 T3170 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5024 T3170 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5025 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5025 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5026 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5026 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5027 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5028 T3170 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5028 T3170 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5029 T3170 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5029 T3170 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5042 T3170 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5047 T3170 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5048 T3170 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5051 T3170 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1
[junit4:junit4]   2> 5052 T3170 oass.SolrIndexSearcher.<init> Opening Searcher@35f635aa main
[junit4:junit4]   2> 5053 T3170 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5054 T3170 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5054 T3170 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5062 T3170 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5125 T3171 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@35f635aa main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5131 T3170 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5132 T3170 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5270 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5271 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63082_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63082/e_vr/g"}
[junit4:junit4]   2> 5277 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63087_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63087/e_vr/g"}
[junit4:junit4]   2> 5277 T3151 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 4
[junit4:junit4]   2> 5278 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5287 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5287 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5287 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6135 T3170 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6135 T3170 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63087/e_vr/g collection:collection1 shard:shard1
[junit4:junit4]   2> 6136 T3170 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6155 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:delete cxid:0x4f zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6156 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6157 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:create cxid:0x50 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6166 T3170 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6166 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6167 T3170 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63087/e_vr/g/collection1/
[junit4:junit4]   2> 6167 T3170 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6167 T3170 oasc.SyncStrategy.syncToMe http://127.0.0.1:63087/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 6168 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63087/e_vr/g/collection1/
[junit4:junit4]   2> 6168 T3170 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6172 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:create cxid:0x5a zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6800 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6818 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6818 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6818 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6848 T3170 oasc.ZkController.register We are http://127.0.0.1:63087/e_vr/g/collection1/ and leader is http://127.0.0.1:63087/e_vr/g/collection1/
[junit4:junit4]   2> 6848 T3170 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63087/e_vr/g
[junit4:junit4]   2> 6848 T3170 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6848 T3170 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6849 T3170 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6850 T3170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6851 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6852 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6852 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7032 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7036 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63090
[junit4:junit4]   2> 7037 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7037 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7038 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914
[junit4:junit4]   2> 7038 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/solr.xml
[junit4:junit4]   2> 7039 T3128 oasc.CoreContainer.<init> New CoreContainer 1662129960
[junit4:junit4]   2> 7040 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/'
[junit4:junit4]   2> 7040 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/'
[junit4:junit4]   2> 7097 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7098 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7099 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7099 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7100 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7100 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7101 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7101 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7102 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7102 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7123 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7143 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 7144 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7145 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7837efa1
[junit4:junit4]   2> 7146 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7146 T3181 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7147 T3181 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 7147 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63091
[junit4:junit4]   2> 7148 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63091
[junit4:junit4]   2> 7155 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850007 with negotiated timeout 20000 for client /140.211.11.196:63091
[junit4:junit4]   2> 7156 T3181 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850007, negotiated timeout = 20000
[junit4:junit4]   2> 7156 T3182 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7837efa1 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7156 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7157 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850007
[junit4:junit4]   2> 7158 T3182 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7158 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63091 which had sessionid 0x13c724f9a850007
[junit4:junit4]   2> 7158 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850007 closed
[junit4:junit4]   2> 7159 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7170 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5313a3ea
[junit4:junit4]   2> 7172 T3183 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7171 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7172 T3183 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 7172 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63092
[junit4:junit4]   2> 7173 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63092
[junit4:junit4]   2> 7174 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850008 with negotiated timeout 20000 for client /140.211.11.196:63092
[junit4:junit4]   2> 7174 T3183 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850008, negotiated timeout = 20000
[junit4:junit4]   2> 7175 T3184 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5313a3ea name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7175 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7176 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7185 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7186 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63090_e_vr%2Fg
[junit4:junit4]   2> 7187 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:63090_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:63090_e_vr%2Fg
[junit4:junit4]   2> 7189 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63090_e_vr%2Fg
[junit4:junit4]   2> 7195 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7196 T3169 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7196 T3156 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7196 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7196 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7198 T3150 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7199 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7206 T3185 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1
[junit4:junit4]   2> 7206 T3185 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7206 T3185 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7207 T3185 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7208 T3185 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1/'
[junit4:junit4]   2> 7209 T3185 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1/lib/README' to classloader
[junit4:junit4]   2> 7210 T3185 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7264 T3185 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7338 T3185 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7439 T3185 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7445 T3185 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8023 T3185 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8027 T3185 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8045 T3185 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/
[junit4:junit4]   2> 8045 T3185 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 8046 T3185 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 8047 T3185 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2 forceNew:false
[junit4:junit4]   2> 8047 T3185 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2
[junit4:junit4]   2> 8048 T3185 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index/
[junit4:junit4]   2> 8048 T3185 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8049 T3185 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index forceNew:false
[junit4:junit4]   2> 8056 T3185 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45893737; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8056 T3185 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8057 T3185 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index
[junit4:junit4]   2> 8058 T3185 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8060 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8060 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8061 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8061 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8062 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8062 T3185 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8063 T3185 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8063 T3185 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8064 T3185 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8065 T3185 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8065 T3185 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8066 T3185 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8066 T3185 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8066 T3185 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8067 T3185 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8068 T3185 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8068 T3185 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8069 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8069 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8070 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8070 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8071 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8072 T3185 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8072 T3185 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8073 T3185 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8073 T3185 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8086 T3185 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8091 T3185 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8092 T3185 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8094 T3185 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2
[junit4:junit4]   2> 8096 T3185 oass.SolrIndexSearcher.<init> Opening Searcher@15e4e139 main
[junit4:junit4]   2> 8097 T3185 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8097 T3185 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8098 T3185 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8105 T3185 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8164 T3186 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15e4e139 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8169 T3185 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8170 T3185 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8330 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8331 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63087_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63087/e_vr/g"}
[junit4:junit4]   2> 8334 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63090_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63090/e_vr/g"}
[junit4:junit4]   2> 8334 T3151 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=4
[junit4:junit4]   2> 8335 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 8347 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8347 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8347 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8347 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9172 T3185 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9172 T3185 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63090/e_vr/g collection:collection1 shard:shard2
[junit4:junit4]   2> 9173 T3185 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9217 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:delete cxid:0x4e zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9218 T3185 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9218 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:create cxid:0x4f zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9229 T3185 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9229 T3185 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9229 T3185 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63090/e_vr/g/collection1/
[junit4:junit4]   2> 9230 T3185 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9230 T3185 oasc.SyncStrategy.syncToMe http://127.0.0.1:63090/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 9230 T3185 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63090/e_vr/g/collection1/
[junit4:junit4]   2> 9230 T3185 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9233 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:create cxid:0x58 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9860 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9906 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9906 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9906 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9906 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9949 T3185 oasc.ZkController.register We are http://127.0.0.1:63090/e_vr/g/collection1/ and leader is http://127.0.0.1:63090/e_vr/g/collection1/
[junit4:junit4]   2> 9949 T3185 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63090/e_vr/g
[junit4:junit4]   2> 9949 T3185 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9949 T3185 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9950 T3185 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9951 T3185 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9952 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9953 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9953 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10139 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10142 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63096
[junit4:junit4]   2> 10143 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10144 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10144 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017
[junit4:junit4]   2> 10145 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/solr.xml
[junit4:junit4]   2> 10145 T3128 oasc.CoreContainer.<init> New CoreContainer 1146688834
[junit4:junit4]   2> 10146 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/'
[junit4:junit4]   2> 10147 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/'
[junit4:junit4]   2> 10204 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10205 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10206 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10206 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10207 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10207 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10208 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10208 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10209 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10209 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10229 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10249 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 10249 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10250 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@15cedd82
[junit4:junit4]   2> 10251 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10252 T3196 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10252 T3196 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 10253 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63097
[junit4:junit4]   2> 10253 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63097
[junit4:junit4]   2> 10255 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850009 with negotiated timeout 20000 for client /140.211.11.196:63097
[junit4:junit4]   2> 10255 T3196 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850009, negotiated timeout = 20000
[junit4:junit4]   2> 10255 T3197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15cedd82 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10256 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10257 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850009
[junit4:junit4]   2> 10273 T3197 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10273 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63097 which had sessionid 0x13c724f9a850009
[junit4:junit4]   2> 10273 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850009 closed
[junit4:junit4]   2> 10274 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10284 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3458ebd0
[junit4:junit4]   2> 10285 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10285 T3198 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10286 T3198 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 10286 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63098
[junit4:junit4]   2> 10286 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63098
[junit4:junit4]   2> 10287 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000a with negotiated timeout 20000 for client /140.211.11.196:63098
[junit4:junit4]   2> 10287 T3198 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000a, negotiated timeout = 20000
[junit4:junit4]   2> 10288 T3199 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3458ebd0 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10288 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10289 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10290 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10297 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63096_e_vr%2Fg
[junit4:junit4]   2> 10298 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:63096_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:63096_e_vr%2Fg
[junit4:junit4]   2> 10299 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63096_e_vr%2Fg
[junit4:junit4]   2> 10301 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10301 T3184 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10302 T3156 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10302 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10302 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10302 T3169 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10302 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10303 T3150 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10310 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10317 T3200 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1
[junit4:junit4]   2> 10317 T3200 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10318 T3200 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10318 T3200 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10319 T3200 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1/'
[junit4:junit4]   2> 10321 T3200 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1/lib/README' to classloader
[junit4:junit4]   2> 10321 T3200 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10377 T3200 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10448 T3200 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10549 T3200 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10555 T3200 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11110 T3200 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11114 T3200 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11132 T3200 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/
[junit4:junit4]   2> 11132 T3200 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 11133 T3200 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 11134 T3200 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3 forceNew:false
[junit4:junit4]   2> 11135 T3200 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3
[junit4:junit4]   2> 11135 T3200 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index/
[junit4:junit4]   2> 11135 T3200 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11136 T3200 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index forceNew:false
[junit4:junit4]   2> 11142 T3200 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a786f7; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11143 T3200 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11144 T3200 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index
[junit4:junit4]   2> 11144 T3200 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11147 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11147 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11147 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11148 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11148 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11149 T3200 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11149 T3200 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11150 T3200 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11150 T3200 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11151 T3200 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11152 T3200 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11152 T3200 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11153 T3200 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11153 T3200 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11153 T3200 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11154 T3200 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11155 T3200 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11155 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11156 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11156 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11157 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11158 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11158 T3200 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11159 T3200 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11159 T3200 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11160 T3200 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11173 T3200 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11178 T3200 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11179 T3200 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11181 T3200 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3
[junit4:junit4]   2> 11183 T3200 oass.SolrIndexSearcher.<init> Opening Searcher@2a2a4855 main
[junit4:junit4]   2> 11184 T3200 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11185 T3200 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11185 T3200 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11192 T3200 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11244 T3201 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2a2a4855 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11249 T3200 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11250 T3200 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11419 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11420 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63090_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63090/e_vr/g"}
[junit4:junit4]   2> 11426 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63096_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63096/e_vr/g"}
[junit4:junit4]   2> 11426 T3151 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=4
[junit4:junit4]   2> 11426 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4:junit4]   2> 11438 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11438 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11438 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11438 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11438 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12253 T3200 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12253 T3200 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63096/e_vr/g collection:collection1 shard:shard3
[junit4:junit4]   2> 12254 T3200 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4:junit4]   2> 12269 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:delete cxid:0x4e zxid:0x9d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard3 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard3
[junit4:junit4]   2> 12270 T3200 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 12270 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:create cxid:0x4f zxid:0x9e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12272 T3200 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 12272 T3200 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 12273 T3200 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63096/e_vr/g/collection1/
[junit4:junit4]   2> 12273 T3200 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 12273 T3200 oasc.SyncStrategy.syncToMe http://127.0.0.1:63096/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 12273 T3200 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63096/e_vr/g/collection1/
[junit4:junit4]   2> 12274 T3200 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4:junit4]   2> 12277 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:create cxid:0x58 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12951 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12993 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12993 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12993 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12993 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12993 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13053 T3200 oasc.ZkController.register We are http://127.0.0.1:63096/e_vr/g/collection1/ and leader is http://127.0.0.1:63096/e_vr/g/collection1/
[junit4:junit4]   2> 13053 T3200 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63096/e_vr/g
[junit4:junit4]   2> 13053 T3200 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 13054 T3200 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 13054 T3200 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13056 T3200 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13056 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 13057 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13058 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13240 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 13243 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:31843
[junit4:junit4]   2> 13244 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13244 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13245 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120
[junit4:junit4]   2> 13245 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/solr.xml
[junit4:junit4]   2> 13246 T3128 oasc.CoreContainer.<init> New CoreContainer 1052215713
[junit4:junit4]   2> 13247 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/'
[junit4:junit4]   2> 13247 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/'
[junit4:junit4]   2> 13304 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 13305 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13306 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 13306 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13307 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13307 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13308 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13308 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13309 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13309 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 13330 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13350 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 13351 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 13352 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3f199de9
[junit4:junit4]   2> 13353 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13353 T3211 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13354 T3211 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 13354 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:61409
[junit4:junit4]   2> 13355 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:61409
[junit4:junit4]   2> 13356 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000b with negotiated timeout 20000 for client /140.211.11.196:61409
[junit4:junit4]   2> 13357 T3211 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000b, negotiated timeout = 20000
[junit4:junit4]   2> 13357 T3212 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f199de9 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13357 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13358 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a85000b
[junit4:junit4]   2> 13368 T3212 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 13368 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:61409 which had sessionid 0x13c724f9a85000b
[junit4:junit4]   2> 13368 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a85000b closed
[junit4:junit4]   2> 13368 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 13380 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@146aced5
[junit4:junit4]   2> 13381 T3213 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13381 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13382 T3213 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 13382 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:48296
[junit4:junit4]   2> 13382 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:48296
[junit4:junit4]   2> 13383 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000c with negotiated timeout 20000 for client /140.211.11.196:48296
[junit4:junit4]   2> 13383 T3213 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000c, negotiated timeout = 20000
[junit4:junit4]   2> 13384 T3214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@146aced5 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13384 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13385 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:create cxid:0x1 zxid:0xae txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13386 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:create cxid:0x2 zxid:0xaf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13395 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31843_e_vr%2Fg
[junit4:junit4]   2> 13396 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:delete cxid:0x4 zxid:0xb0 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:31843_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:31843_e_vr%2Fg
[junit4:junit4]   2> 13398 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31843_e_vr%2Fg
[junit4:junit4]   2> 13401 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13401 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13401 T3184 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13401 T3169 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13401 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13401 T3156 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13402 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13402 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13402 T3150 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13408 T3199 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13409 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13416 T3215 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1
[junit4:junit4]   2> 13417 T3215 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13417 T3215 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13417 T3215 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13419 T3215 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1/'
[junit4:junit4]   2> 13420 T3215 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13420 T3215 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1/lib/README' to classloader
[junit4:junit4]   2> 13476 T3215 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 13548 T3215 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13649 T3215 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13655 T3215 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14208 T3215 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14213 T3215 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14230 T3215 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/
[junit4:junit4]   2> 14230 T3215 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 14231 T3215 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 14232 T3215 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4 forceNew:false
[junit4:junit4]   2> 14233 T3215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4
[junit4:junit4]   2> 14233 T3215 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index/
[junit4:junit4]   2> 14234 T3215 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14234 T3215 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index forceNew:false
[junit4:junit4]   2> 14266 T3215 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@65b69892; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14267 T3215 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14268 T3215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index
[junit4:junit4]   2> 14268 T3215 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14270 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14271 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14271 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14272 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14272 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14273 T3215 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14273 T3215 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14274 T3215 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14274 T3215 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14275 T3215 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14276 T3215 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14276 T3215 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14277 T3215 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14277 T3215 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14277 T3215 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14278 T3215 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14279 T3215 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14279 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14280 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14280 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14281 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14281 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14282 T3215 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14283 T3215 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14283 T3215 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14284 T3215 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14297 T3215 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14302 T3215 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14303 T3215 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14305 T3215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4
[junit4:junit4]   2> 14307 T3215 oass.SolrIndexSearcher.<init> Opening Searcher@3de7fa00 main
[junit4:junit4]   2> 14308 T3215 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14309 T3215 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14309 T3215 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14316 T3215 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14370 T3216 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3de7fa00 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14376 T3215 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14376 T3215 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14534 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14535 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard3",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63096_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63096/e_vr/g"}
[junit4:junit4]   2> 14562 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:31843_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:31843/e_vr/g"}
[junit4:junit4]   2> 14563 T3151 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=4
[junit4:junit4]   2> 14563 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard4
[junit4:junit4]   2> 14581 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15378 T3215 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15378 T3215 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:31843/e_vr/g collection:collection1 shard:shard4
[junit4:junit4]   2> 15379 T3215 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard4/election
[junit4:junit4]   2> 15401 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:delete cxid:0x4e zxid:0xbe txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard4 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard4
[junit4:junit4]   2> 15402 T3215 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 15402 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:create cxid:0x4f zxid:0xbf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15413 T3215 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 15413 T3215 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 15413 T3215 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:31843/e_vr/g/collection1/
[junit4:junit4]   2> 15414 T3215 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 15414 T3215 oasc.SyncStrategy.syncToMe http://127.0.0.1:31843/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 15414 T3215 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:31843/e_vr/g/collection1/
[junit4:junit4]   2> 15415 T3215 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard4
[junit4:junit4]   2> 15417 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:create cxid:0x58 zxid:0xc2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16094 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16112 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16133 T3215 oasc.ZkController.register We are http://127.0.0.1:31843/e_vr/g/collection1/ and leader is http://127.0.0.1:31843/e_vr/g/collection1/
[junit4:junit4]   2> 16133 T3215 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:31843/e_vr/g
[junit4:junit4]   2> 16133 T3215 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 16133 T3215 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 16134 T3215 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16222 T3215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16223 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 16224 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 16224 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16410 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 16414 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62755
[junit4:junit4]   2> 16414 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 16415 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 16416 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288
[junit4:junit4]   2> 16416 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/solr.xml
[junit4:junit4]   2> 16417 T3128 oasc.CoreContainer.<init> New CoreContainer 1715666898
[junit4:junit4]   2> 16418 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/'
[junit4:junit4]   2> 16418 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/'
[junit4:junit4]   2> 16476 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 16477 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 16478 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 16478 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 16479 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 16479 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 16480 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 16480 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 16481 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 16481 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 16501 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 16521 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 16522 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 16523 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@20e8d6d5
[junit4:junit4]   2> 16524 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 16525 T3226 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 16525 T3226 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 16525 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18011
[junit4:junit4]   2> 16526 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18011
[junit4:junit4]   2> 16528 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000d with negotiated timeout 20000 for client /140.211.11.196:18011
[junit4:junit4]   2> 16528 T3226 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000d, negotiated timeout = 20000
[junit4:junit4]   2> 16528 T3227 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20e8d6d5 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 16529 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 16530 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a85000d
[junit4:junit4]   2> 16543 T3227 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 16543 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18011 which had sessionid 0x13c724f9a85000d
[junit4:junit4]   2> 16543 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a85000d closed
[junit4:junit4]   2> 16544 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 16553 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4b9a9539
[junit4:junit4]   2> 16554 T3228 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 16554 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 16555 T3228 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 16555 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49062
[junit4:junit4]   2> 16555 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49062
[junit4:junit4]   2> 16557 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000e with negotiated timeout 20000 for client /140.211.11.196:49062
[junit4:junit4]   2> 16557 T3228 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000e, negotiated timeout = 20000
[junit4:junit4]   2> 16557 T3229 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b9a9539 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 16557 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 16558 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000e type:create cxid:0x1 zxid:0xcf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16560 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000e type:create cxid:0x2 zxid:0xd0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16561 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62755_e_vr%2Fg
[junit4:junit4]   2> 16562 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000e type:delete cxid:0x4 zxid:0xd1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:62755_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:62755_e_vr%2Fg
[junit4:junit4]   2> 16571 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62755_e_vr%2Fg
[junit4:junit4]   2> 16573 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16573 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16574 T3214 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16574 T3169 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16574 T3156 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16574 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 16574 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 16574 T3184 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16575 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 16575 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 16581 T3150 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16581 T3199 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16582 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 16589 T3230 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1
[junit4:junit4]   2> 16590 T3230 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 16590 T3230 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 16590 T3230 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 16592 T3230 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1/'
[junit4:junit4]   2> 16593 T3230 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 16593 T3230 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1/lib/README' to classloader
[junit4:junit4]   2> 16650 T3230 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 16752 T3230 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 16853 T3230 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 16859 T3230 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 17422 T3230 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 17426 T3230 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 17445 T3230 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/
[junit4:junit4]   2> 17445 T3230 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 17445 T3230 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 17446 T3230 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5 forceNew:false
[junit4:junit4]   2> 17447 T3230 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5
[junit4:junit4]   2> 17447 T3230 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index/
[junit4:junit4]   2> 17448 T3230 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 17449 T3230 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index forceNew:false
[junit4:junit4]   2> 17468 T3230 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@50da1982; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17468 T3230 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17469 T3230 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index
[junit4:junit4]   2> 17470 T3230 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 17472 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 17472 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 17473 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 17473 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 17474 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 17474 T3230 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 17474 T3230 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 17475 T3230 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 17475 T3230 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 17476 T3230 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 17477 T3230 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 17477 T3230 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 17478 T3230 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 17478 T3230 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 17478 T3230 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 17479 T3230 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 17479 T3230 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17480 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17480 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17481 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17482 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17482 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17483 T3230 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17483 T3230 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17484 T3230 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 17484 T3230 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 17496 T3230 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 17501 T3230 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 17502 T3230 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 17505 T3230 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5
[junit4:junit4]   2> 17506 T3230 oass.SolrIndexSearcher.<init> Opening Searcher@707caaf1 main
[junit4:junit4]   2> 17507 T3230 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 17507 T3230 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 17508 T3230 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 17537 T3230 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 17616 T3231 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@707caaf1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17621 T3230 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 17621 T3230 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17724 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17725 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard4",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:31843_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:31843/e_vr/g"}
[junit4:junit4]   2> 17734 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62755_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62755/e_vr/g"}
[junit4:junit4]   2> 17734 T3151 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=4
[junit4:junit4]   2> 17734 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 17745 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 18624 T3230 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 18624 T3230 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62755/e_vr/g collection:collection1 shard:shard1
[junit4:junit4]   2> 18653 T3230 oasc.ZkController.register We are http://127.0.0.1:62755/e_vr/g/collection1/ and leader is http://127.0.0.1:63087/e_vr/g/collection1/
[junit4:junit4]   2> 18653 T3230 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62755/e_vr/g
[junit4:junit4]   2> 18653 T3230 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 18654 T3230 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C246 name=collection1 org.apache.solr.core.SolrCore@2af2fa91 url=http://127.0.0.1:62755/e_vr/g/collection1 node=127.0.0.1:62755_e_vr%2Fg C246_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:62755_e_vr%2Fg, base_url=http://127.0.0.1:62755/e_vr/g}
[junit4:junit4]   2> 18654 T3232 C246 P62755 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 18654 T3230 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18654 T3232 C246 P62755 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 18655 T3232 C246 P62755 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 18655 T3232 C246 P62755 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18655 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 18656 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 18656 T3232 C246 P62755 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18657 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18845 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 18848 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50078
[junit4:junit4]   2> 18849 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 18849 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 18850 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725
[junit4:junit4]   2> 18851 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/solr.xml
[junit4:junit4]   2> 18851 T3128 oasc.CoreContainer.<init> New CoreContainer 1070352967
[junit4:junit4]   2> 18852 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/'
[junit4:junit4]   2> 18853 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/'
[junit4:junit4]   2> 18910 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 18910 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 18911 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 18911 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 18912 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 18912 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 18913 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 18914 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 18914 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 18915 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 18934 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 18954 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 18955 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 18955 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@39ceb057
[junit4:junit4]   2> 18956 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18957 T3242 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18957 T3242 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 18958 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25869
[junit4:junit4]   2> 18958 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25869
[junit4:junit4]   2> 18966 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000f with negotiated timeout 20000 for client /140.211.11.196:25869
[junit4:junit4]   2> 18966 T3242 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000f, negotiated timeout = 20000
[junit4:junit4]   2> 18966 T3243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39ceb057 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18966 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18967 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a85000f
[junit4:junit4]   2> 18969 T3243 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 18969 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25869 which had sessionid 0x13c724f9a85000f
[junit4:junit4]   2> 18969 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a85000f closed
[junit4:junit4]   2> 18970 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 18979 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3cb0a0df
[junit4:junit4]   2> 18980 T3244 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18980 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18981 T3244 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 18981 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18741
[junit4:junit4]   2> 18981 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18741
[junit4:junit4]   2> 18983 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850010 with negotiated timeout 20000 for client /140.211.11.196:18741
[junit4:junit4]   2> 18983 T3244 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850010, negotiated timeout = 20000
[junit4:junit4]   2> 18983 T3245 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cb0a0df name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18984 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18984 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850010 type:create cxid:0x1 zxid:0xe1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18993 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850010 type:create cxid:0x2 zxid:0xe2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18995 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50078_e_vr%2Fg
[junit4:junit4]   2> 18995 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850010 type:delete cxid:0x4 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:50078_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:50078_e_vr%2Fg
[junit4:junit4]   2> 18997 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50078_e_vr%2Fg
[junit4:junit4]   2> 19005 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19005 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19005 T3156 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19006 T3184 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19006 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19006 T3214 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19005 T3169 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19006 T3150 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19005 T3229 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19007 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19007 T3229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19006 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19006 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19007 T3199 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19009 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 19015 T3246 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/collection1
[junit4:junit4]   2> 19016 T3246 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 19016 T3246 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 19016 T3246 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 19018 T3246 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/collection1/'
[junit4:junit4]   2> 19019 T3246 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/collection1/lib/README' to classloader
[junit4:junit4]   2> 19019 T3246 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 19075 T3246 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 19147 T3246 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 19248 T3246 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 19254 T3246 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 19257 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19258 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62755_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62755/e_vr/g"}
[junit4:junit4]   2> 19271 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3245 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred -

[...truncated too long message...]

 2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  24) Thread[id=1246, name=searcherExecutor-642-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=1179, name=qtp603328428-1179 Acceptor0 SocketConnector@127.0.0.1:14011, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  26) Thread[id=1192, name=searcherExecutor-598-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  27) Thread[id=1176, name=searcherExecutor-588-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {n_f1=PostingsFormat(name=MockSep), foo_b=PostingsFormat(name=TestBloomFilteredLucene41Postings), cat=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), foo_d=PostingsFormat(name=Asserting), foo_f=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tl1=PostingsFormat(name=MockSep), n_d1=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), rnd_b=PostingsFormat(name=Asserting), intDefault=PostingsFormat(name=MockSep), n_td1=PostingsFormat(name=MockSep), timestamp=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), id=PostingsFormat(name=Asserting), range_facet_sl=PostingsFormat(name=Asserting), range_facet_si=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), oddField_s=PostingsFormat(name=Asserting), sequence_i=PostingsFormat(name=Asserting), name=PostingsFormat(name=Asserting), foo_i=PostingsFormat(name=MockSep), regex_dup_B_s=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), multiDefault=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), n_tf1=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), n_dt1=PostingsFormat(name=MockSep), genre_s=PostingsFormat(name=Asserting), author_t=PostingsFormat(name=MockSep), n_ti1=PostingsFormat(name=Asserting), range_facet_l=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), text=PostingsFormat(name=TestBloomFilteredLucene41Postings), _version_=PostingsFormat(name=MockSep), val_i=PostingsFormat(name=Asserting), SubjectTerms_mfacet=PostingsFormat(name=MockSep), series_t=PostingsFormat(name=MockSep), a_t=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), n_tdt1=PostingsFormat(name=MockSep), regex_dup_A_s=PostingsFormat(name=TestBloomFilteredLucene41Postings), price=PostingsFormat(name=TestBloomFilteredLucene41Postings), other_tl1=PostingsFormat(name=MockSep), n_l1=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), a_si=PostingsFormat(name=MockSep), inStock=PostingsFormat(name=TestBloomFilteredLucene41Postings)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=en, timezone=Atlantic/Reykjavik
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=28,free=172784200,total=427819008
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestSurroundQueryParser, ClusterStateUpdateTest, LegacyHTMLStripCharFilterTest, SynonymTokenizerTest, SpellingQueryConverterTest, DOMUtilTest, TermVectorComponentTest, ClusterStateTest, IndexSchemaTest, ConvertedLegacyTest, SolrIndexConfigTest, TimeZoneUtilsTest, StandardRequestHandlerTest, TestReplicationHandler, TestDistributedGrouping, TestDistributedSearch, TestRTGBase, StatsComponentTest, TestUtils, TestPropInject, TestPluginEnable, TestMergePolicyConfig, DirectUpdateHandlerTest, LukeRequestHandlerTest, DebugComponentTest, SignatureUpdateProcessorFactoryTest, TestLRUCache, PingRequestHandlerTest, TestCSVLoader, PathHierarchyTokenizerFactoryTest, FileUtilsTest, TestSolrXMLSerializer, TestFunctionQuery, SuggesterTSTTest, UUIDFieldTest, TestFoldingMultitermQuery, TestSolrDeletionPolicy2, IndexReaderFactoryTest, PrimUtilsTest, TestRemoteStreaming, OverseerCollectionProcessorTest, PeerSyncTest, TestBinaryField, TestDFRSimilarityFactory, TestDocumentBuilder, LeaderElectionTest, MultiTermTest, ZkNodePropsTest, DisMaxRequestHandlerTest, TermVectorComponentDistributedTest, TestPropInjectDefaults, SuggesterTest, DocumentBuilderTest, ReturnFieldsTest, TestWriterPerf, SchemaVersionSpecificBehaviorTest, TestDocSet, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=73DB5873A286C061 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en -Dtests.timezone=Atlantic/Reykjavik -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J1 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([73DB5873A286C061]:0)
[junit4:junit4] Completed on J1 in 7223.69s, 1 test, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 247 suites, 1032 tests, 1 suite-level error, 2 errors, 23 ignored (5 assumptions)

Total time: 145 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure