You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/01/24 09:13:53 UTC

[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3674 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3674/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([7031C15A2CD49CF9]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([7031C15A2CD49CF9]:0)




Build Log:
[...truncated 9461 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 1 T417 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /gj/cf
[junit4:junit4]   2> 4 T417 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359007985075
[junit4:junit4]   2> 5 T417 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 6 T418 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 8 T418 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 8 T418 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 8 T418 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 8 T418 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 9 T418 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 107 T417 oasc.ZkTestServer.run start zk server on port:51325
[junit4:junit4]   2> 107 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@79d05d5
[junit4:junit4]   2> 108 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 108 T423 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 109 T423 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 109 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:50980
[junit4:junit4]   2> 109 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:50980
[junit4:junit4]   2> 110 T421 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 112 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80000 with negotiated timeout 10000 for client /140.211.11.196:50980
[junit4:junit4]   2> 112 T423 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80000, negotiated timeout = 10000
[junit4:junit4]   2> 112 T424 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79d05d5 name:ZooKeeperConnection Watcher:127.0.0.1:51325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 113 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 113 T417 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 116 T422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6b302db80000
[junit4:junit4]   2> 117 T424 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 117 T417 oaz.ZooKeeper.close Session: 0x13c6b302db80000 closed
[junit4:junit4]   2> 117 T419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:50980 which had sessionid 0x13c6b302db80000
[junit4:junit4]   2> 117 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@12aab346
[junit4:junit4]   2> 118 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 118 T425 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 119 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:47137
[junit4:junit4]   2> 119 T425 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 119 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:47137
[junit4:junit4]   2> 120 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80001 with negotiated timeout 10000 for client /140.211.11.196:47137
[junit4:junit4]   2> 121 T425 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80001, negotiated timeout = 10000
[junit4:junit4]   2> 121 T426 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12aab346 name:ZooKeeperConnection Watcher:127.0.0.1:51325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 121 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 121 T417 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 126 T417 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 135 T417 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 138 T417 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 141 T417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 142 T417 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 152 T417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 153 T417 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 257 T417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 258 T417 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 261 T417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 261 T417 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 264 T417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 265 T417 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 276 T417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 276 T417 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 279 T417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 279 T417 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 282 T417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 283 T417 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 290 T417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 290 T417 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 293 T422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6b302db80001
[junit4:junit4]   2> 294 T426 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 294 T417 oaz.ZooKeeper.close Session: 0x13c6b302db80001 closed
[junit4:junit4]   2> 294 T419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:47137 which had sessionid 0x13c6b302db80001
[junit4:junit4]   2> 421 T417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 425 T417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45036
[junit4:junit4]   2> 426 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 426 T417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 427 T417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359007985364
[junit4:junit4]   2> 427 T417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359007985364/solr.xml
[junit4:junit4]   2> 427 T417 oasc.CoreContainer.<init> New CoreContainer 136714309
[junit4:junit4]   2> 428 T417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359007985364/'
[junit4:junit4]   2> 428 T417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359007985364/'
[junit4:junit4]   2> 467 T417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 467 T417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 467 T417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 468 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 468 T417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 468 T417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 469 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 469 T417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 469 T417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 470 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 484 T417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 499 T417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51325/solr
[junit4:junit4]   2> 500 T417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 500 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5f665b5f
[junit4:junit4]   2> 501 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 501 T436 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 502 T436 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 502 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10043
[junit4:junit4]   2> 503 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10043
[junit4:junit4]   2> 504 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80002 with negotiated timeout 20000 for client /140.211.11.196:10043
[junit4:junit4]   2> 504 T436 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80002, negotiated timeout = 20000
[junit4:junit4]   2> 505 T437 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f665b5f name:ZooKeeperConnection Watcher:127.0.0.1:51325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 505 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 507 T422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6b302db80002
[junit4:junit4]   2> 508 T437 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 508 T419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10043 which had sessionid 0x13c6b302db80002
[junit4:junit4]   2> 508 T417 oaz.ZooKeeper.close Session: 0x13c6b302db80002 closed
[junit4:junit4]   2> 508 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 516 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@453f3251
[junit4:junit4]   2> 517 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 517 T438 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 518 T438 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 518 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:48389
[junit4:junit4]   2> 518 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:48389
[junit4:junit4]   2> 519 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80003 with negotiated timeout 20000 for client /140.211.11.196:48389
[junit4:junit4]   2> 519 T438 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80003, negotiated timeout = 20000
[junit4:junit4]   2> 519 T439 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@453f3251 name:ZooKeeperConnection Watcher:127.0.0.1:51325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 520 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 521 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 530 T417 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 532 T417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45036_gj%2Fcf
[junit4:junit4]   2> 533 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:45036_gj%2Fcf Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:45036_gj%2Fcf
[junit4:junit4]   2> 534 T417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45036_gj%2Fcf
[junit4:junit4]   2> 537 T417 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 549 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 550 T417 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 559 T417 oasc.Overseer.start Overseer (id=89063947310202883-127.0.0.1:45036_gj%2Fcf-n_0000000000) starting
[junit4:junit4]   2> 559 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 560 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 562 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 571 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 572 T441 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 572 T417 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 582 T417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 583 T440 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 587 T442 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359007985364/collection1
[junit4:junit4]   2> 587 T442 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 588 T442 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 588 T442 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 589 T442 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359007985364/collection1/'
[junit4:junit4]   2> 590 T442 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359007985364/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 590 T442 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359007985364/collection1/lib/README' to classloader
[junit4:junit4]   2> 629 T442 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 679 T442 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 780 T442 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 784 T442 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1221 T442 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1228 T442 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1231 T442 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1251 T442 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1255 T442 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1259 T442 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1260 T442 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1262 T442 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359007985364/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data/
[junit4:junit4]   2> 1262 T442 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a6134d4
[junit4:junit4]   2> 1262 T442 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 1291 T442 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data forceNew:false
[junit4:junit4]   2> 1291 T442 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data
[junit4:junit4]   2> 1291 T442 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data/index/
[junit4:junit4]   2> 1292 T442 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1293 T442 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data/index forceNew:false
[junit4:junit4]   2> 1333 T442 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f6a35e1),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1333 T442 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1333 T442 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data/index
[junit4:junit4]   2> 1334 T442 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1335 T442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1335 T442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1336 T442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1336 T442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1336 T442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1337 T442 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1337 T442 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1337 T442 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1338 T442 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1338 T442 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1338 T442 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1339 T442 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1339 T442 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1339 T442 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1339 T442 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1340 T442 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1340 T442 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1340 T442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1340 T442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1341 T442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1341 T442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1341 T442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1342 T442 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1342 T442 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1342 T442 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1343 T442 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1351 T442 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1354 T442 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1356 T442 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1358 T442 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data
[junit4:junit4]   2> 1359 T442 oass.SolrIndexSearcher.<init> Opening Searcher@5efd906e main
[junit4:junit4]   2> 1359 T442 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1360 T442 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1360 T442 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1376 T442 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1423 T443 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5efd906e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1426 T442 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1426 T442 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1427 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2086 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2087 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45036_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45036/gj/cf"}
[junit4:junit4]   2> 2087 T440 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2087 T440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2088 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2112 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2440 T442 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2440 T442 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45036/gj/cf collection:control_collection shard:shard1
[junit4:junit4]   2> 2441 T442 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2461 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2462 T442 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2463 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2473 T442 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2473 T442 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2474 T442 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45036/gj/cf/collection1/
[junit4:junit4]   2> 2474 T442 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2474 T442 oasc.SyncStrategy.syncToMe http://127.0.0.1:45036/gj/cf/collection1/ has no replicas
[junit4:junit4]   2> 2474 T442 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45036/gj/cf/collection1/
[junit4:junit4]   2> 2475 T442 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2480 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3616 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3661 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3713 T442 oasc.ZkController.register We are http://127.0.0.1:45036/gj/cf/collection1/ and leader is http://127.0.0.1:45036/gj/cf/collection1/
[junit4:junit4]   2> 3713 T442 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45036/gj/cf
[junit4:junit4]   2> 3713 T442 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3713 T442 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3714 T442 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3716 T442 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3717 T417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3718 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3718 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3732 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3741 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3dd6dd04
[junit4:junit4]   2> 3741 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3742 T444 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3742 T444 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 3743 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12102
[junit4:junit4]   2> 3743 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12102
[junit4:junit4]   2> 3744 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80004 with negotiated timeout 10000 for client /140.211.11.196:12102
[junit4:junit4]   2> 3744 T444 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80004, negotiated timeout = 10000
[junit4:junit4]   2> 3745 T445 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3dd6dd04 name:ZooKeeperConnection Watcher:127.0.0.1:51325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3745 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3746 T417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3748 T417 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3915 T417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3918 T417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10971
[junit4:junit4]   2> 3919 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3919 T417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3920 T417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359007988818
[junit4:junit4]   2> 3920 T417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359007988818/solr.xml
[junit4:junit4]   2> 3921 T417 oasc.CoreContainer.<init> New CoreContainer 1661917598
[junit4:junit4]   2> 3922 T417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359007988818/'
[junit4:junit4]   2> 3922 T417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359007988818/'
[junit4:junit4]   2> 3977 T417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 3978 T417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3978 T417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 3979 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3979 T417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3979 T417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3980 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3980 T417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3981 T417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3981 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4002 T417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4021 T417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51325/solr
[junit4:junit4]   2> 4022 T417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4022 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1188c915
[junit4:junit4]   2> 4023 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4024 T455 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4025 T455 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 4025 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10587
[junit4:junit4]   2> 4025 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10587
[junit4:junit4]   2> 4027 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80005 with negotiated timeout 20000 for client /140.211.11.196:10587
[junit4:junit4]   2> 4027 T455 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80005, negotiated timeout = 20000
[junit4:junit4]   2> 4028 T456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1188c915 name:ZooKeeperConnection Watcher:127.0.0.1:51325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4028 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4029 T422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6b302db80005
[junit4:junit4]   2> 4037 T456 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4037 T419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10587 which had sessionid 0x13c6b302db80005
[junit4:junit4]   2> 4037 T417 oaz.ZooKeeper.close Session: 0x13c6b302db80005 closed
[junit4:junit4]   2> 4038 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4047 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4e3dfdb8
[junit4:junit4]   2> 4048 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4048 T457 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4049 T457 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 4049 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12769
[junit4:junit4]   2> 4049 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12769
[junit4:junit4]   2> 4050 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80006 with negotiated timeout 20000 for client /140.211.11.196:12769
[junit4:junit4]   2> 4051 T457 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80006, negotiated timeout = 20000
[junit4:junit4]   2> 4051 T458 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e3dfdb8 name:ZooKeeperConnection Watcher:127.0.0.1:51325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4051 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4052 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4054 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4061 T417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10971_gj%2Fcf
[junit4:junit4]   2> 4062 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:10971_gj%2Fcf Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:10971_gj%2Fcf
[junit4:junit4]   2> 4063 T417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10971_gj%2Fcf
[junit4:junit4]   2> 4066 T445 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4066 T439 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4066 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 4075 T417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4083 T459 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359007988818/collection1
[junit4:junit4]   2> 4083 T459 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4083 T459 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4084 T459 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4085 T459 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359007988818/collection1/'
[junit4:junit4]   2> 4086 T459 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359007988818/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4087 T459 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359007988818/collection1/lib/README' to classloader
[junit4:junit4]   2> 4141 T459 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4213 T459 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4314 T459 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4321 T459 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4919 T459 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4927 T459 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4930 T459 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4943 T459 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4947 T459 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4950 T459 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4951 T459 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4952 T459 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359007988818/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/
[junit4:junit4]   2> 4953 T459 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a6134d4
[junit4:junit4]   2> 4953 T459 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 4954 T459 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1 forceNew:false
[junit4:junit4]   2> 4954 T459 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1
[junit4:junit4]   2> 4954 T459 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/index/
[junit4:junit4]   2> 4955 T459 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4956 T459 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/index forceNew:false
[junit4:junit4]   2> 4986 T459 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@627a9569),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4986 T459 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 4986 T459 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/index
[junit4:junit4]   2> 4987 T459 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4988 T459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4989 T459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4989 T459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4989 T459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4990 T459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4990 T459 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4990 T459 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4990 T459 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4991 T459 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4991 T459 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4992 T459 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4992 T459 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4992 T459 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4993 T459 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4993 T459 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4993 T459 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4994 T459 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4994 T459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4994 T459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4995 T459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4995 T459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4996 T459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4996 T459 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4996 T459 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4997 T459 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4997 T459 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5006 T459 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5009 T459 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5010 T459 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5012 T459 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1
[junit4:junit4]   2> 5013 T459 oass.SolrIndexSearcher.<init> Opening Searcher@7461d8f5 main
[junit4:junit4]   2> 5014 T459 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5014 T459 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5014 T459 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5020 T459 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5074 T460 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7461d8f5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5078 T459 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5079 T459 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5174 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5175 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45036_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45036/gj/cf"}
[junit4:junit4]   2> 5178 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10971_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10971/gj/cf"}
[junit4:junit4]   2> 5178 T440 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 5178 T440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5189 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5189 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5190 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6083 T459 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6083 T459 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10971/gj/cf collection:collection1 shard:shard1
[junit4:junit4]   2> 6084 T459 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6101 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6102 T459 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6103 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6113 T459 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6113 T459 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6114 T459 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10971/gj/cf/collection1/
[junit4:junit4]   2> 6114 T459 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6114 T459 oasc.SyncStrategy.syncToMe http://127.0.0.1:10971/gj/cf/collection1/ has no replicas
[junit4:junit4]   2> 6114 T459 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10971/gj/cf/collection1/
[junit4:junit4]   2> 6114 T459 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6119 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6706 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6778 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6778 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6778 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6785 T459 oasc.ZkController.register We are http://127.0.0.1:10971/gj/cf/collection1/ and leader is http://127.0.0.1:10971/gj/cf/collection1/
[junit4:junit4]   2> 6785 T459 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10971/gj/cf
[junit4:junit4]   2> 6785 T459 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6785 T459 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6786 T459 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6813 T459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6815 T417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6815 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6816 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7074 T417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7079 T417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57732
[junit4:junit4]   2> 7080 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7081 T417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7081 T417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359007991902
[junit4:junit4]   2> 7082 T417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359007991902/solr.xml
[junit4:junit4]   2> 7083 T417 oasc.CoreContainer.<init> New CoreContainer 1482837206
[junit4:junit4]   2> 7084 T417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359007991902/'
[junit4:junit4]   2> 7085 T417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359007991902/'
[junit4:junit4]   2> 7173 T417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7174 T417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7175 T417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7176 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7176 T417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7177 T417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7178 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7178 T417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7179 T417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7180 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7209 T417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7239 T417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51325/solr
[junit4:junit4]   2> 7240 T417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7240 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@775f9eaa
[junit4:junit4]   2> 7242 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7242 T470 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7243 T470 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 7244 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57733
[junit4:junit4]   2> 7244 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57733
[junit4:junit4]   2> 7249 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80007 with negotiated timeout 20000 for client /140.211.11.196:57733
[junit4:junit4]   2> 7249 T470 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80007, negotiated timeout = 20000
[junit4:junit4]   2> 7250 T471 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@775f9eaa name:ZooKeeperConnection Watcher:127.0.0.1:51325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7250 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7252 T422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6b302db80007
[junit4:junit4]   2> 7253 T471 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7253 T419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57733 which had sessionid 0x13c6b302db80007
[junit4:junit4]   2> 7253 T417 oaz.ZooKeeper.close Session: 0x13c6b302db80007 closed
[junit4:junit4]   2> 7254 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7269 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@71f8f6e4
[junit4:junit4]   2> 7270 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7271 T472 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7272 T472 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 7272 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57734
[junit4:junit4]   2> 7272 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57734
[junit4:junit4]   2> 7275 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80008 with negotiated timeout 20000 for client /140.211.11.196:57734
[junit4:junit4]   2> 7275 T472 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80008, negotiated timeout = 20000
[junit4:junit4]   2> 7276 T473 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71f8f6e4 name:ZooKeeperConnection Watcher:127.0.0.1:51325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7276 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7278 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7282 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7284 T417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57732_gj%2Fcf
[junit4:junit4]   2> 7285 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57732_gj%2Fcf Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57732_gj%2Fcf
[junit4:junit4]   2> 7286 T417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57732_gj%2Fcf
[junit4:junit4]   2> 7294 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7295 T439 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7295 T445 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7295 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7295 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7297 T458 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7301 T417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7310 T474 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359007991902/collection1
[junit4:junit4]   2> 7310 T474 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7312 T474 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7312 T474 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7314 T474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359007991902/collection1/'
[junit4:junit4]   2> 7315 T474 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359007991902/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7316 T474 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359007991902/collection1/lib/README' to classloader
[junit4:junit4]   2> 7368 T474 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7417 T474 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7518 T474 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7523 T474 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7995 T474 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8003 T474 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8006 T474 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8023 T474 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8028 T474 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8031 T474 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8033 T474 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8034 T474 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359007991902/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/
[junit4:junit4]   2> 8035 T474 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a6134d4
[junit4:junit4]   2> 8035 T474 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 8036 T474 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2 forceNew:false
[junit4:junit4]   2> 8036 T474 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2
[junit4:junit4]   2> 8036 T474 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/index/
[junit4:junit4]   2> 8037 T474 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8038 T474 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/index forceNew:false
[junit4:junit4]   2> 8067 T474 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@12d93aef),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8068 T474 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8068 T474 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/index
[junit4:junit4]   2> 8068 T474 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8070 T474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8070 T474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8071 T474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8071 T474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8071 T474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8071 T474 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8072 T474 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8072 T474 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8073 T474 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8073 T474 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8073 T474 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8074 T474 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8074 T474 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8074 T474 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8075 T474 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8075 T474 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8075 T474 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8076 T474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8076 T474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8077 T474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8077 T474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8077 T474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8078 T474 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8078 T474 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8079 T474 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8079 T474 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8087 T474 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8091 T474 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8092 T474 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8094 T474 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2
[junit4:junit4]   2> 8095 T474 oass.SolrIndexSearcher.<init> Opening Searcher@7824d158 main
[junit4:junit4]   2> 8096 T474 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8096 T474 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8096 T474 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8102 T474 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8150 T475 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7824d158 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8154 T474 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8154 T474 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8317 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8319 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10971_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10971/gj/cf"}
[junit4:junit4]   2> 8345 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57732_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57732/gj/cf"}
[junit4:junit4]   2> 8345 T440 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 8346 T440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 8365 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8365 T473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8365 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8365 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9157 T474 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9157 T474 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57732/gj/cf collection:collection1 shard:shard2
[junit4:junit4]   2> 9159 T474 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9201 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80008 type:delete cxid:0x42 zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9203 T474 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9204 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80008 type:create cxid:0x43 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9214 T474 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9215 T474 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9215 T474 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57732/gj/cf/collection1/
[junit4:junit4]   2> 9215 T474 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9215 T474 oasc.SyncStrategy.syncToMe http://127.0.0.1:57732/gj/cf/collection1/ has no replicas
[junit4:junit4]   2> 9216 T474 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57732/gj/cf/collection1/
[junit4:junit4]   2> 9216 T474 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9222 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db80008 type:create cxid:0x4c zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9881 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9924 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9924 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9924 T473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9924 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9945 T474 oasc.ZkController.register We are http://127.0.0.1:57732/gj/cf/collection1/ and leader is http://127.0.0.1:57732/gj/cf/collection1/
[junit4:junit4]   2> 9945 T474 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57732/gj/cf
[junit4:junit4]   2> 9945 T474 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9946 T474 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9946 T474 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9949 T474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9950 T417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9950 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9951 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10088 T417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10090 T417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57736
[junit4:junit4]   2> 10091 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10091 T417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10092 T417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359007995033
[junit4:junit4]   2> 10092 T417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359007995033/solr.xml
[junit4:junit4]   2> 10092 T417 oasc.CoreContainer.<init> New CoreContainer 1215643479
[junit4:junit4]   2> 10093 T417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359007995033/'
[junit4:junit4]   2> 10093 T417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359007995033/'
[junit4:junit4]   2> 10136 T417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10136 T417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10137 T417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10137 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10137 T417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10138 T417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10138 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10138 T417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10139 T417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10139 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10157 T417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10173 T417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51325/solr
[junit4:junit4]   2> 10174 T417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10174 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6f111770
[junit4:junit4]   2> 10175 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10176 T485 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10176 T485 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 10177 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57737
[junit4:junit4]   2> 10177 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57737
[junit4:junit4]   2> 10179 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db80009 with negotiated timeout 20000 for client /140.211.11.196:57737
[junit4:junit4]   2> 10179 T485 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db80009, negotiated timeout = 20000
[junit4:junit4]   2> 10179 T486 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f111770 name:ZooKeeperConnection Watcher:127.0.0.1:51325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10180 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10181 T422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6b302db80009
[junit4:junit4]   2> 10183 T486 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10183 T419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57737 which had sessionid 0x13c6b302db80009
[junit4:junit4]   2> 10183 T417 oaz.ZooKeeper.close Session: 0x13c6b302db80009 closed
[junit4:junit4]   2> 10184 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10196 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@533a7ed2
[junit4:junit4]   2> 10198 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10198 T487 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10199 T487 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 10199 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57738
[junit4:junit4]   2> 10199 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57738
[junit4:junit4]   2> 10201 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db8000a with negotiated timeout 20000 for client /140.211.11.196:57738
[junit4:junit4]   2> 10201 T487 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db8000a, negotiated timeout = 20000
[junit4:junit4]   2> 10201 T488 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@533a7ed2 name:ZooKeeperConnection Watcher:127.0.0.1:51325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10201 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10202 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db8000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10204 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db8000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10212 T417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57736_gj%2Fcf
[junit4:junit4]   2> 10213 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db8000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57736_gj%2Fcf Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57736_gj%2Fcf
[junit4:junit4]   2> 10222 T417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57736_gj%2Fcf
[junit4:junit4]   2> 10225 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10226 T473 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10226 T445 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10226 T439 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10226 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10226 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10226 T473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10228 T458 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10232 T417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10241 T489 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359007995033/collection1
[junit4:junit4]   2> 10241 T489 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10242 T489 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10242 T489 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10244 T489 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359007995033/collection1/'
[junit4:junit4]   2> 10245 T489 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359007995033/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10245 T489 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359007995033/collection1/lib/README' to classloader
[junit4:junit4]   2> 10288 T489 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10340 T489 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10460 T489 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10465 T489 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10972 T489 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10981 T489 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10985 T489 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11000 T489 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11004 T489 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11009 T489 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11010 T489 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11011 T489 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359007995033/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/
[junit4:junit4]   2> 11011 T489 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a6134d4
[junit4:junit4]   2> 11012 T489 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 11013 T489 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3 forceNew:false
[junit4:junit4]   2> 11013 T489 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 11013 T489 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index/
[junit4:junit4]   2> 11014 T489 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11015 T489 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index forceNew:false
[junit4:junit4]   2> 11019 T489 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2df7c110),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11020 T489 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11020 T489 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index
[junit4:junit4]   2> 11021 T489 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11023 T489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11023 T489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11023 T489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11023 T489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11024 T489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11024 T489 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11024 T489 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11025 T489 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11025 T489 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11026 T489 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11026 T489 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11027 T489 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11027 T489 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11027 T489 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11028 T489 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11028 T489 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11029 T489 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11029 T489 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11029 T489 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11030 T489 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11030 T489 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11031 T489 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11031 T489 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11032 T489 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11032 T489 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11033 T489 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11043 T489 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11046 T489 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11048 T489 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11050 T489 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 11051 T489 oass.SolrIndexSearcher.<init> Opening Searcher@5460fa10 main
[junit4:junit4]   2> 11052 T489 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11052 T489 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11053 T489 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11059 T489 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11119 T490 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5460fa10 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11129 T489 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11130 T489 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11439 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11440 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57732_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57732/gj/cf"}
[junit4:junit4]   2> 11445 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57736_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57736/gj/cf"}
[junit4:junit4]   2> 11445 T440 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 11445 T440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11459 T488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11459 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11459 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11459 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11459 T473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12133 T489 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12133 T489 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57736/gj/cf collection:collection1 shard:shard1
[junit4:junit4]   2> 12141 T489 oasc.ZkController.register We are http://127.0.0.1:57736/gj/cf/collection1/ and leader is http://127.0.0.1:10971/gj/cf/collection1/
[junit4:junit4]   2> 12141 T489 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57736/gj/cf
[junit4:junit4]   2> 12141 T489 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 12142 T489 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C79 name=collection1 org.apache.solr.core.SolrCore@9505713 url=http://127.0.0.1:57736/gj/cf/collection1 node=127.0.0.1:57736_gj%2Fcf C79_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57736_gj%2Fcf, base_url=http://127.0.0.1:57736/gj/cf}
[junit4:junit4]   2> 12142 T491 C79 P57736 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 12143 T491 C79 P57736 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 12143 T491 C79 P57736 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 12143 T491 C79 P57736 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12144 T489 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12145 T491 C79 P57736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12147 T417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12147 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12148 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12381 T417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12384 T417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57740
[junit4:junit4]   2> 12385 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12386 T417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12386 T417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359007997235
[junit4:junit4]   2> 12386 T417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359007997235/solr.xml
[junit4:junit4]   2> 12387 T417 oasc.CoreContainer.<init> New CoreContainer 1561032640
[junit4:junit4]   2> 12388 T417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359007997235/'
[junit4:junit4]   2> 12388 T417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359007997235/'
[junit4:junit4]   2> 12440 T417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 12440 T417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12441 T417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12441 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12442 T417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12442 T417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12442 T417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12443 T417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12443 T417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12444 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12464 T417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12482 T417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51325/solr
[junit4:junit4]   2> 12483 T417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12483 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@78e2119f
[junit4:junit4]   2> 12484 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12484 T501 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12485 T501 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 12485 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57741
[junit4:junit4]   2> 12486 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57741
[junit4:junit4]   2> 12488 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db8000b with negotiated timeout 20000 for client /140.211.11.196:57741
[junit4:junit4]   2> 12488 T501 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db8000b, negotiated timeout = 20000
[junit4:junit4]   2> 12488 T502 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78e2119f name:ZooKeeperConnection Watcher:127.0.0.1:51325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12489 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12490 T422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6b302db8000b
[junit4:junit4]   2> 12506 T502 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12506 T419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57741 which had sessionid 0x13c6b302db8000b
[junit4:junit4]   2> 12506 T417 oaz.ZooKeeper.close Session: 0x13c6b302db8000b closed
[junit4:junit4]   2> 12507 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12518 T417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@11d72f97
[junit4:junit4]   2> 12519 T417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12519 T503 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:51325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12520 T503 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:51325, initiating session
[junit4:junit4]   2> 12520 T419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57742
[junit4:junit4]   2> 12520 T419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57742
[junit4:junit4]   2> 12521 T421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6b302db8000c with negotiated timeout 20000 for client /140.211.11.196:57742
[junit4:junit4]   2> 12522 T503 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:51325, sessionid = 0x13c6b302db8000c, negotiated timeout = 20000
[junit4:junit4]   2> 12522 T504 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11d72f97 name:ZooKeeperConnection Watcher:127.0.0.1:51325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12522 T417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12523 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db8000c type:create cxid:0x1 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12525 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db8000c type:create cxid:0x2 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12526 T417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57740_gj%2Fcf
[junit4:junit4]   2> 12527 T422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6b302db8000c type:delete cxid:0x4 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57740_gj%2Fcf Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57740_gj%2Fcf
[junit4:junit4]   2> 12535 T417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57740_gj%2Fcf
[junit4:junit4]   2> 12538 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12538 T445 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12538 T488 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12539 T473 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12539 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12539 T473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12538 T439 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12539 T488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12539 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12540 T458 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12543 T417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12550 T505 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359007997235/collection1
[junit4:junit4]   2> 12551 T505 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12551 T505 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12552 T505 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12553 T505 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359007997235/collection1/'
[junit4:junit4]   2> 12554 T505 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359007997235/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12555 T505 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359007997235/collection1/lib/README' to classloader
[junit4:junit4]   2> 12622 T505 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12691 T505 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12793 T505 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12799 T505 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12975 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12976 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57736_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57736/gj/cf"}
[junit4:junit4]   2> 12990 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12990 T488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12990 T504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12990 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12990 T473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12990 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13408 T505 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13420 T505 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13437 T505 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13454 T505 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13459 T505 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13464 T505 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13465 T505 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13466 T505 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359007997235/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/
[junit4:junit4]   2> 13467 T505 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a6134d4
[junit4:junit4]   2> 13467 T505 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 13468 T505 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4 forceNew:false
[junit4:junit4]   2> 13469 T505 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 13469 T505 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index/
[junit4:junit4]   2> 13470 T505 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13471 T505 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index forceNew:false
[junit4:junit4]   2> 13477 T505 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d64f83e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13477 T505 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13478 T505 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index
[junit4:junit4]   2> 13478 T505 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13480 T505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13481 T505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13481 T505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13481 T505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13482 T505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13482 T505 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13483 T505 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13483 T505 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13484 T505 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13485 T505 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13485 T505 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13486 T505 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13486 T505 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13486 T505 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13487 T505 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13487 T505 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13488 T505 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13489 T505 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13489 T505 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13490 T505 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13490 T505 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13491 T505 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13491 T505 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13492 T505 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13493 T505 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13493 T505 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13506 T505 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13511 T505 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13512 T505 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13515 T505 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 13516 T505 oass.SolrIndexSearcher.<init> Opening Searcher@56107940 main
[junit4:junit4]   2> 13517 T505 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13518 T505 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13518 T505 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13525 T505 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13575 T506 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@56107940 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13581 T505 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13581 T505 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14499 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14500 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57740_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57740/gj/cf"}
[junit4:junit4]   2> 14500 T440 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14500 T440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 14547 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14547 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14547 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14547 T473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14547 T488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14547 T504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14584 T505 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 14584 T505 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57740/gj/cf collection:collection1 shard:shard2
[junit4:junit4]   2> 14588 T505 oasc.ZkController.register We are http://127.0.0.1:57740/gj/cf/collection1/ and leader is http://127.0.0.1:57732/gj/cf/collection1/
[junit4:junit4]   2> 14588 T505 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57740/gj/cf
[junit4:junit4]   2> 14588 T505 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 14588 T505 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C80 name=collection1 org.apache.solr.core.SolrCore@17f103f0 url=http://127.0.0.1:57740/gj/cf/collection1 node=127.0.0.1:57740_gj%2Fcf C80_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57740_gj%2Fcf, base_url=http://127.0.0.1:57740/gj/cf}
[junit4:junit4]   2> 14588 T507 C80 P57740 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 14589 T505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14590 T507 C80 P57740 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 14590 T507 C80 P57740 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 14590 T507 C80 P57740 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14590 T417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14591 T417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14592 T507 C80 P57740 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14592 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14608 T417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14609 T417 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 14610 T417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C79_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57736_gj%2Fcf, base_url=http://127.0.0.1:57736/gj/cf}
[junit4:junit4]   2> 15169 T491 C79 P57736 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:10971/gj/cf/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15169 T491 C79 P57736 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57736/gj/cf START replicas=[http://127.0.0.1:10971/gj/cf/collection1/] nUpdates=100
[junit4:junit4]   2> 15169 T491 C79 P57736 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15170 T491 C79 P57736 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15170 T491 C79 P57736 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15170 T491 C79 P57736 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15170 T491 C79 P57736 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15171 T491 C79 P57736 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:10971/gj/cf/collection1/. core=collection1
[junit4:junit4]   2> 15171 T491 C79 P57736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C81 name=collection1 org.apache.solr.core.SolrCore@20030dce url=http://127.0.0.1:10971/gj/cf/collection1 node=127.0.0.1:10971_gj%2Fcf C81_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:10971_gj%2Fcf, base_url=http://127.0.0.1:10971/gj/cf, leader=true}
[junit4:junit4]   2> 15182 T449 C81 P10971 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15185 T450 C81 P10971 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15186 T450 C81 P10971 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1
[junit4:junit4]   2> 15190 T450 C81 P10971 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@627a9569),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15191 T450 C81 P10971 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15219 T450 C81 P10971 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@627a9569),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@627a9569),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15220 T450 C81 P10971 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15221 T450 C81 P10971 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1
[junit4:junit4]   2> 15222 T450 C81 P10971 oass.SolrIndexSearcher.<init> Opening Searcher@351e647e realtime
[junit4:junit4]   2> 15222 T450 C81 P10971 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15222 T450 C81 P10971 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 37
[junit4:junit4]   2> 15223 T491 C79 P57736 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15224 T491 C79 P57736 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15227 T451 C81 P10971 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15227 T451 C81 P10971 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15228 T491 C79 P57736 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15228 T491 C79 P57736 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15228 T491 C79 P57736 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15231 T451 C81 P10971 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1
[junit4:junit4]   2> 15232 T451 C81 P10971 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1/index
[junit4:junit4]   2> 15232 T451 C81 P10971 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15233 T491 C79 P57736 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15234 T491 C79 P57736 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index.20130124011320304 forceNew:false
[junit4:junit4]   2> 15235 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 15235 T491 C79 P57736 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index.20130124011320304 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1814840d) fullCopy=true
[junit4:junit4]   2> 15239 T451 C81 P10971 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 15241 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index
[junit4:junit4]   2> 15241 T491 C79 P57736 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15241 T491 C79 P57736 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130124011320304
[junit4:junit4]   2> 15242 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 15243 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 15243 T491 C79 P57736 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index.20130124011320304
[junit4:junit4]   2> 15245 T491 C79 P57736 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index.20130124011320304 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1814840d),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15246 T491 C79 P57736 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15247 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 15247 T491 C79 P57736 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15247 T491 C79 P57736 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15247 T491 C79 P57736 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 15248 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index.20130124011320304
[junit4:junit4]   2> 15248 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 15250 T491 C79 P57736 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index.20130124011320304 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1814840d),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15250 T491 C79 P57736 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15251 T491 C79 P57736 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15251 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 15251 T491 C79 P57736 oass.SolrIndexSearcher.<init> Opening Searcher@2c219a21 main
[junit4:junit4]   2> 15253 T490 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c219a21 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15254 T490 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index
[junit4:junit4]   2> 15254 T491 C79 P57736 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15256 T491 C79 P57736 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index.20130124011320304 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1814840d),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index.20130124011320304 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1814840d),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 15256 T491 C79 P57736 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 15256 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 15257 T491 C79 P57736 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15257 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index.20130124011320304
[junit4:junit4]   2> 15257 T491 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3/index
[junit4:junit4]   2> 15257 T491 C79 P57736 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15257 T491 C79 P57736 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15257 T491 C79 P57736 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15257 T491 C79 P57736 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15273 T491 C79 P57736 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 15612 T417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16053 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16054 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57740_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57740/gj/cf"}
[junit4:junit4]   2> 16084 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57736_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57736/gj/cf"}
[junit4:junit4]   2> 16096 T488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16096 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16096 T504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16096 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16096 T473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16096 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16615 T417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17618 T417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C80_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57740_gj%2Fcf, base_url=http://127.0.0.1:57740/gj/cf}
[junit4:junit4]   2> 18612 T507 C80 P57740 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57732/gj/cf/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18612 T507 C80 P57740 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57740/gj/cf START replicas=[http://127.0.0.1:57732/gj/cf/collection1/] nUpdates=100
[junit4:junit4]   2> 18612 T507 C80 P57740 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18613 T507 C80 P57740 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18613 T507 C80 P57740 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18613 T507 C80 P57740 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18613 T507 C80 P57740 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18613 T507 C80 P57740 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57732/gj/cf/collection1/. core=collection1
[junit4:junit4]   2> 18613 T507 C80 P57740 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18620 T417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C82 name=collection1 org.apache.solr.core.SolrCore@75e99238 url=http://127.0.0.1:57732/gj/cf/collection1 node=127.0.0.1:57732_gj%2Fcf C82_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57732_gj%2Fcf, base_url=http://127.0.0.1:57732/gj/cf, leader=true}
[junit4:junit4]   2> 18621 T464 C82 P57732 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18624 T465 C82 P57732 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18625 T465 C82 P57732 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2
[junit4:junit4]   2> 18629 T465 C82 P57732 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@12d93aef),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18630 T465 C82 P57732 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18634 T465 C82 P57732 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@12d93aef),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@12d93aef),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18634 T465 C82 P57732 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18635 T465 C82 P57732 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2
[junit4:junit4]   2> 18636 T465 C82 P57732 oass.SolrIndexSearcher.<init> Opening Searcher@79b9d133 realtime
[junit4:junit4]   2> 18636 T465 C82 P57732 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18636 T465 C82 P57732 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4]   2> 18637 T507 C80 P57740 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18638 T507 C80 P57740 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18639 T466 C82 P57732 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18640 T466 C82 P57732 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18641 T507 C80 P57740 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18641 T507 C80 P57740 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18641 T507 C80 P57740 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18643 T466 C82 P57732 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2
[junit4:junit4]   2> 18644 T466 C82 P57732 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2/index
[junit4:junit4]   2> 18645 T466 C82 P57732 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18645 T507 C80 P57740 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18646 T507 C80 P57740 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index.20130124011323716 forceNew:false
[junit4:junit4]   2> 18647 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 18648 T507 C80 P57740 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index.20130124011323716 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b2a085f) fullCopy=true
[junit4:junit4]   2> 18651 T466 C82 P57732 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 18653 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index
[junit4:junit4]   2> 18653 T507 C80 P57740 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18654 T507 C80 P57740 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130124011323716
[junit4:junit4]   2> 18655 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 18655 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 18655 T507 C80 P57740 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index.20130124011323716
[junit4:junit4]   2> 18658 T507 C80 P57740 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index.20130124011323716 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b2a085f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18658 T507 C80 P57740 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18659 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 18659 T507 C80 P57740 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18660 T507 C80 P57740 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18660 T507 C80 P57740 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18660 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index.20130124011323716
[junit4:junit4]   2> 18661 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 18663 T507 C80 P57740 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index.20130124011323716 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b2a085f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18663 T507 C80 P57740 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18663 T507 C80 P57740 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18664 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 18664 T507 C80 P57740 oass.SolrIndexSearcher.<init> Opening Searcher@69b1d6b main
[junit4:junit4]   2> 18666 T506 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@69b1d6b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18666 T506 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index
[junit4:junit4]   2> 18666 T507 C80 P57740 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18669 T507 C80 P57740 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index.20130124011323716 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b2a085f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index.20130124011323716 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b2a085f),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18669 T507 C80 P57740 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18670 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 18670 T507 C80 P57740 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18670 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index.20130124011323716
[junit4:junit4]   2> 18670 T507 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4/index
[junit4:junit4]   2> 18670 T507 C80 P57740 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18670 T507 C80 P57740 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18671 T507 C80 P57740 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18671 T507 C80 P57740 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18678 T507 C80 P57740 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19111 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19112 T440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57740_gj%2Fcf",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57740/gj/cf"}
[junit4:junit4]   2> 19119 T458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19119 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19119 T473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19119 T504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19119 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19119 T488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19623 T417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19625 T417 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C83 name=collection1 org.apache.solr.core.SolrCore@1b94c231 url=http://127.0.0.1:45036/gj/cf/collection1 node=127.0.0.1:45036_gj%2Fcf C83_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:45036_gj%2Fcf, base_url=http://127.0.0.1:45036/gj/cf, leader=true}
[junit4:junit4]   2> 19634 T429 C83 P45036 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data
[junit4:junit4]   2> 19638 T429 C83 P45036 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f6a35e1),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19638 T429 C83 P45036 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19639 T429 C83 P45036 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/control/data
[junit4:junit4]   2> 19640 T429 C83 P45036 oass.SolrIndexSearcher.<init> Opening Searcher@4922fcf4 realtime
[junit4:junit4]   2> 19640 T429 C83 P45036 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425023177540501504)} 0 7
[junit4:junit4]   2> 19644 T467 C82 P57732 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty2
[junit4:junit4]   2> 19649 T452 C81 P10971 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty1
[junit4:junit4]   2>  C80_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57740_gj%2Fcf, base_url=http://127.0.0.1:57740/gj/cf}
[junit4:junit4]   2> 19654 T494 C80 P57740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty4
[junit4:junit4]   2> 19654 T494 C80 P57740 /update {update.distrib=FROMLEADER&_version_=-1425023177549938688&update.from=http://127.0.0.1:57732/gj/cf/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425023177549938688)} 0 2
[junit4:junit4]   2>  C79_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57736_gj%2Fcf, base_url=http://127.0.0.1:57736/gj/cf}
[junit4:junit4]   2> 19658 T478 C79 P57736 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359007985075/jetty3
[junit4:junit4]   2> 19658 T478 C79 P57736 /update {update.distrib=FROMLEADER&_version_=-1425023177555181568&update.from=http://127.0.0.1:10971/gj/cf/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425023177555181568)} 0 2
[junit4:junit4]   2> 19659 T452 C81 P10971 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425023177555181568)} 0 12
[junit4:junit4]   2> 19660 T467 C82 P57732 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425023177549938688)} 0 18
[junit4:junit4]   2> 19672 T429 C83 P45036 /update {wt=javabin&version=2} {add=[1 (1425023177575104512)]} 0 6
[junit4:junit4]   2> 19687 T478 C79 P57736 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425023177587687424)]} 0 3
[junit4:junit4]   2> 19688 T452 C81 P10971 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1425023177587687424)]} 0 10
[junit4:junit4]   2> 19688 T467 C82 P57732 /update {wt=javabin&version=2} {add=[1]} 0 13
[junit4:junit4]   2> 19695 T429 C83 P45036 /update {wt=javabin&version=2} {add=[2 (1425023177602367488)]} 0 3
[junit4:junit4]   2> 19710 T494 C80 P57740 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425023177608658944)]} 0 5
[junit4:junit4]   2> 19711 T467 C82 P57732 /update {wt=javabin&version=2} {add=[2 (1425023177608658944)]} 0 13
[junit4:junit4]   2> 19716 T429 C83 P45036 /update {wt=javabin&version=2} {add=[3 (1425023177624387584)]} 0 3
[junit4:junit4]   2> 19726 T494 C80 P57740 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425023177629630464)]} 0 2
[junit4:junit4]   2> 19726 T467 C82 P57732 /update {wt=javabin&version=2} {add=[3 (1425023177629630464)]} 0 8
[junit4:junit4]   2> 19731 T429 C83 P45036 /update {wt=javabin&version=2} {add=[4 (1425023177641164800)]} 0 2
[junit4:junit4]   2> 19740 T478 C79 P57736 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425023177645359104)]} 0 2
[junit4:junit4]   2> 19740 T453 C81 P10971 /update {wt=javabin&version=2} {add=[4 (1425023177645359104)]} 0 7
[junit4:junit4]   2> 19745 T429 C83 P45036 /update {wt=javabin&version=2} {add=[5 (1425023177654796288)]} 0 2
[junit4:junit4]   2> 19759 T494 C80 P57740 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425023177662136320)]} 0 2
[junit4:junit4]   2> 19759 T468 C82 P57732 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1425023177662136320)]} 0 10
[junit4:junit4]   2> 19761 T453 C81 P10971 /update {wt=javabin&version=2} {add=[5]} 0 14
[junit4:junit4]   2> 19766 T429 C83 P45036 /update {wt=javabin&version=2} {add=[6 (1425023177676816384)]} 0 2
[junit4:junit4]   2> 19780 T494 C80 P57740 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425023177685204992)]} 0 2
[junit4:junit4]   2> 19780 T468 C82 P57732 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1425023177685204992)]} 0 9
[junit4:junit4]   2> 19782 T453 C81 P10971 /update {wt=javabin&version=2} {add=[6]} 0 14
[junit4:junit4]   2> 19788 T429 C83 P45036 /update {wt=javabin&version=2} {add=[7 (1425023177699885056)]} 0 3
[junit4:junit4]   2> 19797 T494 C80 P57740 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425023177705127936)]} 0 2
[junit4:junit4]   2> 19797 T467 C82 P57732 /update {wt=javabin&version=2} {add=[7 (1425023177705127936)]} 0 6
[junit4:junit4]   2> 19803 T429 C83 P45036 /update {wt=javabin&version=2} {add=[8 (1425023177715613696)]} 0 3
[junit4:junit4]   2> 19815 T478 C79 P57736 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425023177724002304)]} 0 2
[junit4:junit4]   2> 19815 T452 C81 P10971 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1425023177724002304)]} 0 6
[junit4:junit4]   2> 19816 T467 C82 P57732 /update {wt=javabin&version=2} {add=[8]} 0 10
[junit4:junit4]   2> 19820 T429 C83 P45036 /update {wt=javabin&version=2} {add=[9 (1425023177733439488)]} 0 3
[junit4:junit4]   2> 19835 T494 C80 P57740 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425023177742876672)]} 0 2
[junit4:junit4]   2> 19835 T468 C82 P57732 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1425023177742876672)]} 0 8
[junit4:junit4]   2> 19837 T453 C81 P10971 /update {wt=javabin&version=2} {add=[9]} 0 14
[junit4:junit4]   2> 19857 T429 C83 P45036 /update {wt=javabin&version=2} {add=[10 (1425023177757556736)]} 0 17
[junit4:junit4]   2> 19872 T478 C79 P57736 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425023177782722560)]} 0 3
[junit4:junit4]   2> 19872 T452 C81 P10971 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1425023177782722560)]} 0 8
[junit4:junit4]   2> 19873 T467 C82 P57732 /update {wt=javabin&version=2} {add=[10]} 0 13
[junit4:junit4]   2> 19880 T429 C83 P45036 /update {wt=javabin&version=2} {add=[11 (1425023177795305472)]} 0 4
[junit4:junit4]   2> 19892 T478 C79 P57736 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1425023177803694080)]} 0 3
[junit4:junit4]   2> 19892 T452 C81 P10971 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1425023177803694080)]} 0 8
[junit4:junit4]   2> 19893 T467 C82 P57732 /update {wt=javabin&version=2} {add=[11]} 0 12
[junit4:junit4]   2> 19898 T429 C83 P45036 /update {wt=javabin&version=2} {add=[12 (1425023177816276992)]} 0 2
[junit4:junit4]   2> 19909 T478 C79 P57736 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1425023177821519872)]} 0 3
[junit4:junit4]   2> 19909 T453 C81 P10971 /update {wt=javabin&version=2} {add=[12 (1425023177821519872)]} 0 8
[junit4:junit4]   2> 19915 T429 C83 P45036 /update {wt=javabin&version=2} {add=[13 (1425023177834102784)]} 0 2
[junit4:junit4]   2> 19929 T478 C79 P57736 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1425023177842491392)]} 0 3
[junit4:junit4]   2> 19930 T452 C81 P10971 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1425023177842491392)]} 0 9
[junit4:junit4]   2> 19930 T467 C82 P57732 /update {wt=javabin&version=2} {add=[13]} 0 12
[junit4:junit4]   2> 19936 T429 C83 P45036 /update {wt=javabin&version=2} {add=[14 (1425023177856122880)]} 0 2
[junit4:junit4]   2> 19949 T478 C79 P57736 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1425023177864511488)]} 0 2
[junit4:junit4]   2> 19950 T452 C81 P10971 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1425023177864511488)]} 0 8
[junit4:junit4]   2> 19951 T467 C82 P57732 /update {wt=javabin&version=2} {add=[14]} 0 12
[junit4:junit4]   2> 19956 T429 C83 P45036 /update {wt=javabin&version=2} {add=[15 (1425023177877094400)]} 0 2
[junit4:junit4]   2> 19969 T478 C79 P57736 /update {distrib.from=http://127.0.0.1:10971/gj/cf/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1425023177885483008)]} 0 2
[junit4:junit4]   2> 19970 T452 C81 P10971 /update {distrib.from=http://127.0.0.1:57732/gj/cf/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1425023177885483008)]} 0 8
[jun

[...truncated too long message...]

util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  37) Thread[id=545, name=searcherExecutor-555-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  38) Thread[id=790, name=searcherExecutor-686-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  39) Thread[id=784, name=searcherExecutor-678-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  40) Thread[id=553, name=searcherExecutor-573-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  41) Thread[id=608, name=searcherExecutor-611-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  42) Thread[id=794, name=qtp1349418906-794, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.ByteArrayBuffer.readFrom(ByteArrayBuffer.java:375)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.bio.StreamEndPoint.fill(StreamEndPoint.java:141)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.fill(SocketConnector.java:227)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.fill(HttpParser.java:1035)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:280)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  43) Thread[id=487, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[7031C15A2CD49CF9]-SendThread(localhost:51325), state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=hr_HR, timezone=America/Montreal
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=46,free=268025784,total=423624704
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestPHPSerializedResponseWriter, TestGroupingSearch, TestFiltering, CopyFieldTest, TestPseudoReturnFields, LoggingHandlerTest, TestReload, TestConfig, FastVectorHighlighterTest, CircularListTest, SearchHandlerTest, CollectionsAPIDistributedZkTest, AlternateDirectoryTest, TestLMJelinekMercerSimilarityFactory, SortByFunctionTest, BinaryUpdateRequestHandlerTest, TestBadConfig, UniqFieldsUpdateProcessorFactoryTest, DirectSolrConnectionTest, UpdateRequestProcessorFactoryTest, TestStressRecovery, ResourceLoaderTest, TestWordDelimiterFilterFactory, SpellCheckComponentTest, SimpleFacetsTest, FileBasedSpellCheckerTest, TestBinaryResponseWriter, SpellCheckCollatorTest, TestElisionMultitermQuery, TestAnalyzedSuggestions, TestCoreContainer, ScriptEngineTest, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=7031C15A2CD49CF9 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hr_HR -Dtests.timezone=America/Montreal -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([7031C15A2CD49CF9]:0)
[junit4:junit4] Completed on J0 in 7223.90s, 1 test, 2 errors <<< FAILURES!

[...truncated 95 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 247 suites, 1032 tests, 1 suite-level error, 1 error, 128 ignored (3 assumptions)

Total time: 145 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3677 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3677/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.ShardRoutingTest.testDistribSearch

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:63087/e_vr/g/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:63087/e_vr/g/collection1
	at __randomizedtesting.SeedInfo.seed([73DB5873A286C061:F23DD66BD5D9A05D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
	at org.apache.solr.cloud.ShardRoutingTest.doTestNumRequests(ShardRoutingTest.java:224)
	at org.apache.solr.cloud.ShardRoutingTest.doTest(ShardRoutingTest.java:120)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
	... 47 more


FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([73DB5873A286C061]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([73DB5873A286C061]:0)




Build Log:
[...truncated 9323 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ShardRoutingTest
[junit4:junit4]   2> 0 T3128 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /e_vr/g
[junit4:junit4]   2> 5 T3128 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-ShardRoutingTest-1359127485054
[junit4:junit4]   2> 6 T3128 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T3129 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T3129 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T3129 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T3129 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T3129 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T3129 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T3128 oasc.ZkTestServer.run start zk server on port:63079
[junit4:junit4]   2> 108 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1687cd23
[junit4:junit4]   2> 109 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T3134 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T3134 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 110 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63080
[junit4:junit4]   2> 111 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63080
[junit4:junit4]   2> 111 T3132 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 113 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850000 with negotiated timeout 10000 for client /140.211.11.196:63080
[junit4:junit4]   2> 113 T3134 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850000, negotiated timeout = 10000
[junit4:junit4]   2> 114 T3135 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1687cd23 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 114 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 115 T3128 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 117 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850000
[junit4:junit4]   2> 129 T3135 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 130 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63080 which had sessionid 0x13c724f9a850000
[junit4:junit4]   2> 129 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850000 closed
[junit4:junit4]   2> 130 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2aaaab79
[junit4:junit4]   2> 131 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 131 T3136 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 132 T3136 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 132 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63081
[junit4:junit4]   2> 132 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63081
[junit4:junit4]   2> 133 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850001 with negotiated timeout 10000 for client /140.211.11.196:63081
[junit4:junit4]   2> 133 T3136 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850001, negotiated timeout = 10000
[junit4:junit4]   2> 134 T3137 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2aaaab79 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 134 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 135 T3128 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 143 T3128 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 146 T3128 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 148 T3128 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 151 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 152 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 167 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 167 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 274 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 275 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 278 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 278 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 281 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 282 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 285 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 285 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 288 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 289 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 296 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 297 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 300 T3128 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 301 T3128 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 304 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850001
[junit4:junit4]   2> 305 T3137 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 305 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63081 which had sessionid 0x13c724f9a850001
[junit4:junit4]   2> 305 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850001 closed
[junit4:junit4]   2> 474 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 479 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63082
[junit4:junit4]   2> 480 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 481 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 481 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354
[junit4:junit4]   2> 482 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/solr.xml
[junit4:junit4]   2> 482 T3128 oasc.CoreContainer.<init> New CoreContainer 1413403077
[junit4:junit4]   2> 483 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/'
[junit4:junit4]   2> 483 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/'
[junit4:junit4]   2> 540 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 541 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 542 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 542 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 543 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 543 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 544 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 544 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 545 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 545 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 567 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 586 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 587 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 588 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@a00f995
[junit4:junit4]   2> 589 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 589 T3147 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 590 T3147 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 590 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63083
[junit4:junit4]   2> 591 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63083
[junit4:junit4]   2> 593 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850002 with negotiated timeout 20000 for client /140.211.11.196:63083
[junit4:junit4]   2> 593 T3147 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850002, negotiated timeout = 20000
[junit4:junit4]   2> 594 T3148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a00f995 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 594 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 595 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850002
[junit4:junit4]   2> 596 T3148 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 596 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63083 which had sessionid 0x13c724f9a850002
[junit4:junit4]   2> 596 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850002 closed
[junit4:junit4]   2> 597 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 608 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@35ff1f6f
[junit4:junit4]   2> 609 T3149 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 609 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 610 T3149 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 610 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63084
[junit4:junit4]   2> 610 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63084
[junit4:junit4]   2> 611 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850003 with negotiated timeout 20000 for client /140.211.11.196:63084
[junit4:junit4]   2> 612 T3149 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850003, negotiated timeout = 20000
[junit4:junit4]   2> 612 T3150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35ff1f6f name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 612 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 614 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 623 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 625 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63082_e_vr%2Fg
[junit4:junit4]   2> 626 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:63082_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:63082_e_vr%2Fg
[junit4:junit4]   2> 627 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63082_e_vr%2Fg
[junit4:junit4]   2> 636 T3128 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 649 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 650 T3128 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 657 T3128 oasc.Overseer.start Overseer (id=89071778860957699-127.0.0.1:63082_e_vr%2Fg-n_0000000000) starting
[junit4:junit4]   2> 657 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 659 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 660 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 662 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 673 T3152 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 673 T3128 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 676 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 678 T3151 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 683 T3153 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1
[junit4:junit4]   2> 683 T3153 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 684 T3153 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 684 T3153 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 685 T3153 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1/'
[junit4:junit4]   2> 686 T3153 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 687 T3153 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1/lib/README' to classloader
[junit4:junit4]   2> 742 T3153 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 813 T3153 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 913 T3153 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 919 T3153 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1501 T3153 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1505 T3153 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1522 T3153 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-controljetty-1359127485354/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/
[junit4:junit4]   2> 1523 T3153 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 1523 T3153 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1525 T3153 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data forceNew:false
[junit4:junit4]   2> 1525 T3153 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data
[junit4:junit4]   2> 1526 T3153 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index/
[junit4:junit4]   2> 1526 T3153 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1527 T3153 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index forceNew:false
[junit4:junit4]   2> 1534 T3153 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@23d0ecc3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1534 T3153 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1535 T3153 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data/index
[junit4:junit4]   2> 1536 T3153 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1538 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1538 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1538 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1539 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1539 T3153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1540 T3153 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1540 T3153 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1541 T3153 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1541 T3153 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1542 T3153 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1542 T3153 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1543 T3153 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1543 T3153 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1543 T3153 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1544 T3153 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1544 T3153 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1545 T3153 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1545 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1546 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1546 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1547 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1547 T3153 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1548 T3153 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1548 T3153 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1549 T3153 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1549 T3153 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1562 T3153 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1567 T3153 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1568 T3153 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1570 T3153 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/control/data
[junit4:junit4]   2> 1572 T3153 oass.SolrIndexSearcher.<init> Opening Searcher@55fb709c main
[junit4:junit4]   2> 1573 T3153 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1574 T3153 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1574 T3153 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1591 T3153 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1647 T3154 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@55fb709c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1652 T3153 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1652 T3153 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1653 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x5c zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2181 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2181 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63082_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63082/e_vr/g"}
[junit4:junit4]   2> 2182 T3151 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2182 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2182 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x64 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2201 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2666 T3153 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2666 T3153 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63082/e_vr/g collection:control_collection shard:shard1
[junit4:junit4]   2> 2667 T3153 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2690 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:delete cxid:0x7e zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2691 T3153 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2691 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x7f zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2703 T3153 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2703 T3153 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2703 T3153 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63082/e_vr/g/collection1/
[junit4:junit4]   2> 2704 T3153 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2704 T3153 oasc.SyncStrategy.syncToMe http://127.0.0.1:63082/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 2704 T3153 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63082/e_vr/g/collection1/
[junit4:junit4]   2> 2705 T3153 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2714 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850003 type:create cxid:0x89 zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3705 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3757 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3787 T3153 oasc.ZkController.register We are http://127.0.0.1:63082/e_vr/g/collection1/ and leader is http://127.0.0.1:63082/e_vr/g/collection1/
[junit4:junit4]   2> 3787 T3153 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63082/e_vr/g
[junit4:junit4]   2> 3787 T3153 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3787 T3153 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3788 T3153 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3789 T3153 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3790 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3791 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3791 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3805 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3814 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7319a2de
[junit4:junit4]   2> 3815 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3815 T3155 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3815 T3155 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 3816 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63086
[junit4:junit4]   2> 3816 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63086
[junit4:junit4]   2> 3817 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850004 with negotiated timeout 10000 for client /140.211.11.196:63086
[junit4:junit4]   2> 3817 T3155 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850004, negotiated timeout = 10000
[junit4:junit4]   2> 3818 T3156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7319a2de name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3818 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3819 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3820 T3128 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3986 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3990 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63087
[junit4:junit4]   2> 3991 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3992 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3992 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869
[junit4:junit4]   2> 3993 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/solr.xml
[junit4:junit4]   2> 3993 T3128 oasc.CoreContainer.<init> New CoreContainer 1255630714
[junit4:junit4]   2> 3994 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/'
[junit4:junit4]   2> 3995 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/'
[junit4:junit4]   2> 4052 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4053 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4053 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4054 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4054 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4055 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4055 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4056 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4056 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4057 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4076 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4096 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 4096 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4097 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@63f5739
[junit4:junit4]   2> 4098 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4098 T3166 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4099 T3166 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 4099 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63088
[junit4:junit4]   2> 4100 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63088
[junit4:junit4]   2> 4108 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850005 with negotiated timeout 20000 for client /140.211.11.196:63088
[junit4:junit4]   2> 4108 T3166 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850005, negotiated timeout = 20000
[junit4:junit4]   2> 4108 T3167 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63f5739 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4109 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4110 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850005
[junit4:junit4]   2> 4111 T3167 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4111 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63088 which had sessionid 0x13c724f9a850005
[junit4:junit4]   2> 4111 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850005 closed
[junit4:junit4]   2> 4112 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4121 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@43c50499
[junit4:junit4]   2> 4122 T3168 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4122 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4123 T3168 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 4123 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63089
[junit4:junit4]   2> 4123 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63089
[junit4:junit4]   2> 4125 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850006 with negotiated timeout 20000 for client /140.211.11.196:63089
[junit4:junit4]   2> 4125 T3168 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850006, negotiated timeout = 20000
[junit4:junit4]   2> 4125 T3169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43c50499 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4125 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4126 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4133 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4141 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63087_e_vr%2Fg
[junit4:junit4]   2> 4141 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:63087_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:63087_e_vr%2Fg
[junit4:junit4]   2> 4143 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63087_e_vr%2Fg
[junit4:junit4]   2> 4145 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4146 T3156 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4147 T3150 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4153 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4160 T3170 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1
[junit4:junit4]   2> 4160 T3170 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4161 T3170 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4161 T3170 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4162 T3170 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1/'
[junit4:junit4]   2> 4164 T3170 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4164 T3170 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1/lib/README' to classloader
[junit4:junit4]   2> 4219 T3170 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4290 T3170 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4391 T3170 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4397 T3170 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4978 T3170 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4983 T3170 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5001 T3170 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty1-1359127488869/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/
[junit4:junit4]   2> 5002 T3170 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 5002 T3170 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 5003 T3170 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1 forceNew:false
[junit4:junit4]   2> 5004 T3170 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1
[junit4:junit4]   2> 5004 T3170 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index/
[junit4:junit4]   2> 5005 T3170 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5006 T3170 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index forceNew:false
[junit4:junit4]   2> 5012 T3170 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b5485d6; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5012 T3170 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5013 T3170 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1/index
[junit4:junit4]   2> 5014 T3170 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5016 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5017 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5017 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5017 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5018 T3170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5018 T3170 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5019 T3170 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5019 T3170 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5020 T3170 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5021 T3170 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5021 T3170 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5022 T3170 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5022 T3170 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5023 T3170 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5023 T3170 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5024 T3170 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5024 T3170 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5025 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5025 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5026 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5026 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5027 T3170 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5028 T3170 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5028 T3170 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5029 T3170 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5029 T3170 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5042 T3170 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5047 T3170 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5048 T3170 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5051 T3170 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty1
[junit4:junit4]   2> 5052 T3170 oass.SolrIndexSearcher.<init> Opening Searcher@35f635aa main
[junit4:junit4]   2> 5053 T3170 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5054 T3170 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5054 T3170 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5062 T3170 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5125 T3171 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@35f635aa main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5131 T3170 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5132 T3170 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5270 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5271 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63082_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63082/e_vr/g"}
[junit4:junit4]   2> 5277 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63087_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63087/e_vr/g"}
[junit4:junit4]   2> 5277 T3151 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 4
[junit4:junit4]   2> 5278 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5287 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5287 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5287 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6135 T3170 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6135 T3170 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63087/e_vr/g collection:collection1 shard:shard1
[junit4:junit4]   2> 6136 T3170 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6155 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:delete cxid:0x4f zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6156 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6157 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:create cxid:0x50 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6166 T3170 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6166 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6167 T3170 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63087/e_vr/g/collection1/
[junit4:junit4]   2> 6167 T3170 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6167 T3170 oasc.SyncStrategy.syncToMe http://127.0.0.1:63087/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 6168 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63087/e_vr/g/collection1/
[junit4:junit4]   2> 6168 T3170 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6172 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850006 type:create cxid:0x5a zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6800 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6818 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6818 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6818 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6848 T3170 oasc.ZkController.register We are http://127.0.0.1:63087/e_vr/g/collection1/ and leader is http://127.0.0.1:63087/e_vr/g/collection1/
[junit4:junit4]   2> 6848 T3170 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63087/e_vr/g
[junit4:junit4]   2> 6848 T3170 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6848 T3170 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6849 T3170 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6850 T3170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6851 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6852 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6852 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7032 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7036 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63090
[junit4:junit4]   2> 7037 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7037 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7038 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914
[junit4:junit4]   2> 7038 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/solr.xml
[junit4:junit4]   2> 7039 T3128 oasc.CoreContainer.<init> New CoreContainer 1662129960
[junit4:junit4]   2> 7040 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/'
[junit4:junit4]   2> 7040 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/'
[junit4:junit4]   2> 7097 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7098 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7099 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7099 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7100 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7100 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7101 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7101 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7102 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7102 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7123 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7143 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 7144 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7145 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7837efa1
[junit4:junit4]   2> 7146 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7146 T3181 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7147 T3181 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 7147 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63091
[junit4:junit4]   2> 7148 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63091
[junit4:junit4]   2> 7155 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850007 with negotiated timeout 20000 for client /140.211.11.196:63091
[junit4:junit4]   2> 7156 T3181 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850007, negotiated timeout = 20000
[junit4:junit4]   2> 7156 T3182 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7837efa1 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7156 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7157 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850007
[junit4:junit4]   2> 7158 T3182 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7158 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63091 which had sessionid 0x13c724f9a850007
[junit4:junit4]   2> 7158 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850007 closed
[junit4:junit4]   2> 7159 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7170 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5313a3ea
[junit4:junit4]   2> 7172 T3183 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7171 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7172 T3183 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 7172 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63092
[junit4:junit4]   2> 7173 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63092
[junit4:junit4]   2> 7174 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850008 with negotiated timeout 20000 for client /140.211.11.196:63092
[junit4:junit4]   2> 7174 T3183 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850008, negotiated timeout = 20000
[junit4:junit4]   2> 7175 T3184 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5313a3ea name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7175 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7176 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7185 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7186 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63090_e_vr%2Fg
[junit4:junit4]   2> 7187 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:63090_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:63090_e_vr%2Fg
[junit4:junit4]   2> 7189 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63090_e_vr%2Fg
[junit4:junit4]   2> 7195 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7196 T3169 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7196 T3156 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7196 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7196 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7198 T3150 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7199 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7206 T3185 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1
[junit4:junit4]   2> 7206 T3185 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7206 T3185 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7207 T3185 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7208 T3185 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1/'
[junit4:junit4]   2> 7209 T3185 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1/lib/README' to classloader
[junit4:junit4]   2> 7210 T3185 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7264 T3185 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7338 T3185 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7439 T3185 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7445 T3185 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8023 T3185 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8027 T3185 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8045 T3185 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty2-1359127491914/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/
[junit4:junit4]   2> 8045 T3185 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 8046 T3185 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 8047 T3185 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2 forceNew:false
[junit4:junit4]   2> 8047 T3185 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2
[junit4:junit4]   2> 8048 T3185 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index/
[junit4:junit4]   2> 8048 T3185 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8049 T3185 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index forceNew:false
[junit4:junit4]   2> 8056 T3185 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45893737; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8056 T3185 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8057 T3185 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2/index
[junit4:junit4]   2> 8058 T3185 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8060 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8060 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8061 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8061 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8062 T3185 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8062 T3185 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8063 T3185 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8063 T3185 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8064 T3185 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8065 T3185 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8065 T3185 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8066 T3185 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8066 T3185 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8066 T3185 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8067 T3185 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8068 T3185 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8068 T3185 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8069 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8069 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8070 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8070 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8071 T3185 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8072 T3185 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8072 T3185 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8073 T3185 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8073 T3185 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8086 T3185 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8091 T3185 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8092 T3185 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8094 T3185 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty2
[junit4:junit4]   2> 8096 T3185 oass.SolrIndexSearcher.<init> Opening Searcher@15e4e139 main
[junit4:junit4]   2> 8097 T3185 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8097 T3185 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8098 T3185 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8105 T3185 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8164 T3186 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15e4e139 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8169 T3185 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8170 T3185 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8330 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8331 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63087_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63087/e_vr/g"}
[junit4:junit4]   2> 8334 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63090_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63090/e_vr/g"}
[junit4:junit4]   2> 8334 T3151 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=4
[junit4:junit4]   2> 8335 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 8347 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8347 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8347 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8347 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9172 T3185 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9172 T3185 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63090/e_vr/g collection:collection1 shard:shard2
[junit4:junit4]   2> 9173 T3185 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9217 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:delete cxid:0x4e zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9218 T3185 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9218 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:create cxid:0x4f zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9229 T3185 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9229 T3185 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9229 T3185 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63090/e_vr/g/collection1/
[junit4:junit4]   2> 9230 T3185 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9230 T3185 oasc.SyncStrategy.syncToMe http://127.0.0.1:63090/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 9230 T3185 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63090/e_vr/g/collection1/
[junit4:junit4]   2> 9230 T3185 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9233 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850008 type:create cxid:0x58 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9860 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9906 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9906 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9906 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9906 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9949 T3185 oasc.ZkController.register We are http://127.0.0.1:63090/e_vr/g/collection1/ and leader is http://127.0.0.1:63090/e_vr/g/collection1/
[junit4:junit4]   2> 9949 T3185 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63090/e_vr/g
[junit4:junit4]   2> 9949 T3185 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9949 T3185 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9950 T3185 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9951 T3185 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9952 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9953 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9953 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10139 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10142 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63096
[junit4:junit4]   2> 10143 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10144 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10144 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017
[junit4:junit4]   2> 10145 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/solr.xml
[junit4:junit4]   2> 10145 T3128 oasc.CoreContainer.<init> New CoreContainer 1146688834
[junit4:junit4]   2> 10146 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/'
[junit4:junit4]   2> 10147 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/'
[junit4:junit4]   2> 10204 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10205 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10206 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10206 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10207 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10207 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10208 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10208 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10209 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10209 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10229 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10249 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 10249 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10250 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@15cedd82
[junit4:junit4]   2> 10251 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10252 T3196 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10252 T3196 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 10253 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63097
[junit4:junit4]   2> 10253 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63097
[junit4:junit4]   2> 10255 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850009 with negotiated timeout 20000 for client /140.211.11.196:63097
[junit4:junit4]   2> 10255 T3196 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850009, negotiated timeout = 20000
[junit4:junit4]   2> 10255 T3197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15cedd82 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10256 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10257 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a850009
[junit4:junit4]   2> 10273 T3197 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10273 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63097 which had sessionid 0x13c724f9a850009
[junit4:junit4]   2> 10273 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a850009 closed
[junit4:junit4]   2> 10274 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10284 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3458ebd0
[junit4:junit4]   2> 10285 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10285 T3198 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10286 T3198 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 10286 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63098
[junit4:junit4]   2> 10286 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63098
[junit4:junit4]   2> 10287 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000a with negotiated timeout 20000 for client /140.211.11.196:63098
[junit4:junit4]   2> 10287 T3198 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000a, negotiated timeout = 20000
[junit4:junit4]   2> 10288 T3199 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3458ebd0 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10288 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10289 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10290 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10297 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63096_e_vr%2Fg
[junit4:junit4]   2> 10298 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:63096_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:63096_e_vr%2Fg
[junit4:junit4]   2> 10299 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63096_e_vr%2Fg
[junit4:junit4]   2> 10301 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10301 T3184 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10302 T3156 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10302 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10302 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10302 T3169 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10302 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10303 T3150 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10310 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10317 T3200 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1
[junit4:junit4]   2> 10317 T3200 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10318 T3200 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10318 T3200 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10319 T3200 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1/'
[junit4:junit4]   2> 10321 T3200 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1/lib/README' to classloader
[junit4:junit4]   2> 10321 T3200 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10377 T3200 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10448 T3200 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10549 T3200 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10555 T3200 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11110 T3200 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11114 T3200 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11132 T3200 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty3-1359127495017/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/
[junit4:junit4]   2> 11132 T3200 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 11133 T3200 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 11134 T3200 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3 forceNew:false
[junit4:junit4]   2> 11135 T3200 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3
[junit4:junit4]   2> 11135 T3200 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index/
[junit4:junit4]   2> 11135 T3200 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11136 T3200 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index forceNew:false
[junit4:junit4]   2> 11142 T3200 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a786f7; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11143 T3200 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11144 T3200 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3/index
[junit4:junit4]   2> 11144 T3200 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11147 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11147 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11147 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11148 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11148 T3200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11149 T3200 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11149 T3200 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11150 T3200 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11150 T3200 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11151 T3200 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11152 T3200 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11152 T3200 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11153 T3200 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11153 T3200 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11153 T3200 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11154 T3200 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11155 T3200 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11155 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11156 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11156 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11157 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11158 T3200 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11158 T3200 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11159 T3200 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11159 T3200 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11160 T3200 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11173 T3200 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11178 T3200 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11179 T3200 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11181 T3200 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty3
[junit4:junit4]   2> 11183 T3200 oass.SolrIndexSearcher.<init> Opening Searcher@2a2a4855 main
[junit4:junit4]   2> 11184 T3200 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11185 T3200 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11185 T3200 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11192 T3200 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11244 T3201 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2a2a4855 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11249 T3200 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11250 T3200 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11419 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11420 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63090_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63090/e_vr/g"}
[junit4:junit4]   2> 11426 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63096_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63096/e_vr/g"}
[junit4:junit4]   2> 11426 T3151 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=4
[junit4:junit4]   2> 11426 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4:junit4]   2> 11438 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11438 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11438 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11438 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11438 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12253 T3200 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12253 T3200 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63096/e_vr/g collection:collection1 shard:shard3
[junit4:junit4]   2> 12254 T3200 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4:junit4]   2> 12269 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:delete cxid:0x4e zxid:0x9d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard3 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard3
[junit4:junit4]   2> 12270 T3200 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 12270 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:create cxid:0x4f zxid:0x9e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12272 T3200 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 12272 T3200 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 12273 T3200 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63096/e_vr/g/collection1/
[junit4:junit4]   2> 12273 T3200 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 12273 T3200 oasc.SyncStrategy.syncToMe http://127.0.0.1:63096/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 12273 T3200 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63096/e_vr/g/collection1/
[junit4:junit4]   2> 12274 T3200 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4:junit4]   2> 12277 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000a type:create cxid:0x58 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12951 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12993 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12993 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12993 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12993 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12993 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13053 T3200 oasc.ZkController.register We are http://127.0.0.1:63096/e_vr/g/collection1/ and leader is http://127.0.0.1:63096/e_vr/g/collection1/
[junit4:junit4]   2> 13053 T3200 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63096/e_vr/g
[junit4:junit4]   2> 13053 T3200 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 13054 T3200 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 13054 T3200 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13056 T3200 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13056 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 13057 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13058 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13240 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 13243 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:31843
[junit4:junit4]   2> 13244 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13244 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13245 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120
[junit4:junit4]   2> 13245 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/solr.xml
[junit4:junit4]   2> 13246 T3128 oasc.CoreContainer.<init> New CoreContainer 1052215713
[junit4:junit4]   2> 13247 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/'
[junit4:junit4]   2> 13247 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/'
[junit4:junit4]   2> 13304 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 13305 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13306 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 13306 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13307 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13307 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13308 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13308 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13309 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13309 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 13330 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13350 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 13351 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 13352 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3f199de9
[junit4:junit4]   2> 13353 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13353 T3211 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13354 T3211 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 13354 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:61409
[junit4:junit4]   2> 13355 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:61409
[junit4:junit4]   2> 13356 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000b with negotiated timeout 20000 for client /140.211.11.196:61409
[junit4:junit4]   2> 13357 T3211 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000b, negotiated timeout = 20000
[junit4:junit4]   2> 13357 T3212 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f199de9 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13357 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13358 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a85000b
[junit4:junit4]   2> 13368 T3212 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 13368 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:61409 which had sessionid 0x13c724f9a85000b
[junit4:junit4]   2> 13368 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a85000b closed
[junit4:junit4]   2> 13368 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 13380 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@146aced5
[junit4:junit4]   2> 13381 T3213 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13381 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13382 T3213 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 13382 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:48296
[junit4:junit4]   2> 13382 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:48296
[junit4:junit4]   2> 13383 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000c with negotiated timeout 20000 for client /140.211.11.196:48296
[junit4:junit4]   2> 13383 T3213 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000c, negotiated timeout = 20000
[junit4:junit4]   2> 13384 T3214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@146aced5 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13384 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13385 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:create cxid:0x1 zxid:0xae txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13386 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:create cxid:0x2 zxid:0xaf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13395 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31843_e_vr%2Fg
[junit4:junit4]   2> 13396 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:delete cxid:0x4 zxid:0xb0 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:31843_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:31843_e_vr%2Fg
[junit4:junit4]   2> 13398 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31843_e_vr%2Fg
[junit4:junit4]   2> 13401 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13401 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13401 T3184 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13401 T3169 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13401 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13401 T3156 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13402 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13402 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13402 T3150 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13408 T3199 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13409 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13416 T3215 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1
[junit4:junit4]   2> 13417 T3215 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13417 T3215 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13417 T3215 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13419 T3215 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1/'
[junit4:junit4]   2> 13420 T3215 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13420 T3215 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1/lib/README' to classloader
[junit4:junit4]   2> 13476 T3215 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 13548 T3215 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13649 T3215 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13655 T3215 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14208 T3215 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14213 T3215 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14230 T3215 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty4-1359127498120/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/
[junit4:junit4]   2> 14230 T3215 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 14231 T3215 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 14232 T3215 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4 forceNew:false
[junit4:junit4]   2> 14233 T3215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4
[junit4:junit4]   2> 14233 T3215 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index/
[junit4:junit4]   2> 14234 T3215 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14234 T3215 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index forceNew:false
[junit4:junit4]   2> 14266 T3215 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@65b69892; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14267 T3215 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14268 T3215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4/index
[junit4:junit4]   2> 14268 T3215 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14270 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14271 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14271 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14272 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14272 T3215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14273 T3215 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14273 T3215 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14274 T3215 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14274 T3215 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14275 T3215 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14276 T3215 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14276 T3215 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14277 T3215 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14277 T3215 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14277 T3215 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14278 T3215 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14279 T3215 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14279 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14280 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14280 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14281 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14281 T3215 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14282 T3215 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14283 T3215 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14283 T3215 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14284 T3215 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14297 T3215 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14302 T3215 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14303 T3215 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14305 T3215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty4
[junit4:junit4]   2> 14307 T3215 oass.SolrIndexSearcher.<init> Opening Searcher@3de7fa00 main
[junit4:junit4]   2> 14308 T3215 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14309 T3215 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14309 T3215 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14316 T3215 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14370 T3216 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3de7fa00 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14376 T3215 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14376 T3215 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14534 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14535 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard3",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63096_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63096/e_vr/g"}
[junit4:junit4]   2> 14562 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:31843_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:31843/e_vr/g"}
[junit4:junit4]   2> 14563 T3151 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=4
[junit4:junit4]   2> 14563 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard4
[junit4:junit4]   2> 14581 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14581 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15378 T3215 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15378 T3215 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:31843/e_vr/g collection:collection1 shard:shard4
[junit4:junit4]   2> 15379 T3215 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard4/election
[junit4:junit4]   2> 15401 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:delete cxid:0x4e zxid:0xbe txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard4 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard4
[junit4:junit4]   2> 15402 T3215 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 15402 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:create cxid:0x4f zxid:0xbf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15413 T3215 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 15413 T3215 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 15413 T3215 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:31843/e_vr/g/collection1/
[junit4:junit4]   2> 15414 T3215 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 15414 T3215 oasc.SyncStrategy.syncToMe http://127.0.0.1:31843/e_vr/g/collection1/ has no replicas
[junit4:junit4]   2> 15414 T3215 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:31843/e_vr/g/collection1/
[junit4:junit4]   2> 15415 T3215 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard4
[junit4:junit4]   2> 15417 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000c type:create cxid:0x58 zxid:0xc2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16094 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16112 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16112 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16133 T3215 oasc.ZkController.register We are http://127.0.0.1:31843/e_vr/g/collection1/ and leader is http://127.0.0.1:31843/e_vr/g/collection1/
[junit4:junit4]   2> 16133 T3215 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:31843/e_vr/g
[junit4:junit4]   2> 16133 T3215 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 16133 T3215 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 16134 T3215 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16222 T3215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16223 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 16224 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 16224 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16410 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 16414 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62755
[junit4:junit4]   2> 16414 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 16415 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 16416 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288
[junit4:junit4]   2> 16416 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/solr.xml
[junit4:junit4]   2> 16417 T3128 oasc.CoreContainer.<init> New CoreContainer 1715666898
[junit4:junit4]   2> 16418 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/'
[junit4:junit4]   2> 16418 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/'
[junit4:junit4]   2> 16476 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 16477 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 16478 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 16478 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 16479 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 16479 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 16480 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 16480 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 16481 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 16481 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 16501 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 16521 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 16522 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 16523 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@20e8d6d5
[junit4:junit4]   2> 16524 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 16525 T3226 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 16525 T3226 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 16525 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18011
[junit4:junit4]   2> 16526 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18011
[junit4:junit4]   2> 16528 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000d with negotiated timeout 20000 for client /140.211.11.196:18011
[junit4:junit4]   2> 16528 T3226 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000d, negotiated timeout = 20000
[junit4:junit4]   2> 16528 T3227 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20e8d6d5 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 16529 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 16530 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a85000d
[junit4:junit4]   2> 16543 T3227 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 16543 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18011 which had sessionid 0x13c724f9a85000d
[junit4:junit4]   2> 16543 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a85000d closed
[junit4:junit4]   2> 16544 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 16553 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4b9a9539
[junit4:junit4]   2> 16554 T3228 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 16554 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 16555 T3228 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 16555 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49062
[junit4:junit4]   2> 16555 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49062
[junit4:junit4]   2> 16557 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000e with negotiated timeout 20000 for client /140.211.11.196:49062
[junit4:junit4]   2> 16557 T3228 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000e, negotiated timeout = 20000
[junit4:junit4]   2> 16557 T3229 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b9a9539 name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 16557 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 16558 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000e type:create cxid:0x1 zxid:0xcf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16560 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000e type:create cxid:0x2 zxid:0xd0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16561 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62755_e_vr%2Fg
[junit4:junit4]   2> 16562 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a85000e type:delete cxid:0x4 zxid:0xd1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:62755_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:62755_e_vr%2Fg
[junit4:junit4]   2> 16571 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62755_e_vr%2Fg
[junit4:junit4]   2> 16573 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16573 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16574 T3214 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16574 T3169 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16574 T3156 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16574 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 16574 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 16574 T3184 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16575 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 16575 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 16581 T3150 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16581 T3199 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 16582 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 16589 T3230 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1
[junit4:junit4]   2> 16590 T3230 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 16590 T3230 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 16590 T3230 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 16592 T3230 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1/'
[junit4:junit4]   2> 16593 T3230 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 16593 T3230 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1/lib/README' to classloader
[junit4:junit4]   2> 16650 T3230 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 16752 T3230 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 16853 T3230 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 16859 T3230 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 17422 T3230 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 17426 T3230 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 17445 T3230 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty5-1359127501288/collection1/, dataDir=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/
[junit4:junit4]   2> 17445 T3230 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c9f4521
[junit4:junit4]   2> 17445 T3230 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 17446 T3230 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5 forceNew:false
[junit4:junit4]   2> 17447 T3230 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5
[junit4:junit4]   2> 17447 T3230 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index/
[junit4:junit4]   2> 17448 T3230 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 17449 T3230 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index forceNew:false
[junit4:junit4]   2> 17468 T3230 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@50da1982; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17468 T3230 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17469 T3230 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5/index
[junit4:junit4]   2> 17470 T3230 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 17472 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 17472 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 17473 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 17473 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 17474 T3230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 17474 T3230 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 17474 T3230 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 17475 T3230 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 17475 T3230 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 17476 T3230 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 17477 T3230 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 17477 T3230 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 17478 T3230 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 17478 T3230 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 17478 T3230 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 17479 T3230 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 17479 T3230 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17480 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17480 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17481 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17482 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17482 T3230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17483 T3230 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17483 T3230 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17484 T3230 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 17484 T3230 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 17496 T3230 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 17501 T3230 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 17502 T3230 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 17505 T3230 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-1359127485054/jetty5
[junit4:junit4]   2> 17506 T3230 oass.SolrIndexSearcher.<init> Opening Searcher@707caaf1 main
[junit4:junit4]   2> 17507 T3230 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 17507 T3230 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 17508 T3230 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 17537 T3230 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 17616 T3231 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@707caaf1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17621 T3230 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 17621 T3230 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17724 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17725 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard4",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:31843_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:31843/e_vr/g"}
[junit4:junit4]   2> 17734 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62755_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62755/e_vr/g"}
[junit4:junit4]   2> 17734 T3151 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=4
[junit4:junit4]   2> 17734 T3151 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 17745 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17745 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 18624 T3230 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 18624 T3230 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62755/e_vr/g collection:collection1 shard:shard1
[junit4:junit4]   2> 18653 T3230 oasc.ZkController.register We are http://127.0.0.1:62755/e_vr/g/collection1/ and leader is http://127.0.0.1:63087/e_vr/g/collection1/
[junit4:junit4]   2> 18653 T3230 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62755/e_vr/g
[junit4:junit4]   2> 18653 T3230 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 18654 T3230 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C246 name=collection1 org.apache.solr.core.SolrCore@2af2fa91 url=http://127.0.0.1:62755/e_vr/g/collection1 node=127.0.0.1:62755_e_vr%2Fg C246_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:62755_e_vr%2Fg, base_url=http://127.0.0.1:62755/e_vr/g}
[junit4:junit4]   2> 18654 T3232 C246 P62755 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 18654 T3230 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18654 T3232 C246 P62755 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 18655 T3232 C246 P62755 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 18655 T3232 C246 P62755 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18655 T3128 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 18656 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 18656 T3232 C246 P62755 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18657 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18845 T3128 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 18848 T3128 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50078
[junit4:junit4]   2> 18849 T3128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 18849 T3128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 18850 T3128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725
[junit4:junit4]   2> 18851 T3128 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/solr.xml
[junit4:junit4]   2> 18851 T3128 oasc.CoreContainer.<init> New CoreContainer 1070352967
[junit4:junit4]   2> 18852 T3128 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/'
[junit4:junit4]   2> 18853 T3128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/'
[junit4:junit4]   2> 18910 T3128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 18910 T3128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 18911 T3128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 18911 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 18912 T3128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 18912 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 18913 T3128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 18914 T3128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 18914 T3128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 18915 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 18934 T3128 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 18954 T3128 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63079/solr
[junit4:junit4]   2> 18955 T3128 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 18955 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@39ceb057
[junit4:junit4]   2> 18956 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18957 T3242 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18957 T3242 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 18958 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25869
[junit4:junit4]   2> 18958 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25869
[junit4:junit4]   2> 18966 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a85000f with negotiated timeout 20000 for client /140.211.11.196:25869
[junit4:junit4]   2> 18966 T3242 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a85000f, negotiated timeout = 20000
[junit4:junit4]   2> 18966 T3243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39ceb057 name:ZooKeeperConnection Watcher:127.0.0.1:63079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18966 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18967 T3133 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c724f9a85000f
[junit4:junit4]   2> 18969 T3243 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 18969 T3130 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25869 which had sessionid 0x13c724f9a85000f
[junit4:junit4]   2> 18969 T3128 oaz.ZooKeeper.close Session: 0x13c724f9a85000f closed
[junit4:junit4]   2> 18970 T3128 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 18979 T3128 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63079/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3cb0a0df
[junit4:junit4]   2> 18980 T3244 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:63079. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18980 T3128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18981 T3244 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63079, initiating session
[junit4:junit4]   2> 18981 T3130 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18741
[junit4:junit4]   2> 18981 T3130 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18741
[junit4:junit4]   2> 18983 T3132 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c724f9a850010 with negotiated timeout 20000 for client /140.211.11.196:18741
[junit4:junit4]   2> 18983 T3244 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:63079, sessionid = 0x13c724f9a850010, negotiated timeout = 20000
[junit4:junit4]   2> 18983 T3245 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cb0a0df name:ZooKeeperConnection Watcher:127.0.0.1:63079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18984 T3128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18984 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850010 type:create cxid:0x1 zxid:0xe1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18993 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850010 type:create cxid:0x2 zxid:0xe2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18995 T3128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50078_e_vr%2Fg
[junit4:junit4]   2> 18995 T3133 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c724f9a850010 type:delete cxid:0x4 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:50078_e_vr%2Fg Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:50078_e_vr%2Fg
[junit4:junit4]   2> 18997 T3128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50078_e_vr%2Fg
[junit4:junit4]   2> 19005 T3150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19005 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19005 T3156 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19006 T3184 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19006 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19006 T3214 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19005 T3169 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19006 T3150 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19005 T3229 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19007 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19007 T3229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19006 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19006 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19007 T3199 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19009 T3128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 19015 T3246 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/collection1
[junit4:junit4]   2> 19016 T3246 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 19016 T3246 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 19016 T3246 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 19018 T3246 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/collection1/'
[junit4:junit4]   2> 19019 T3246 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/collection1/lib/README' to classloader
[junit4:junit4]   2> 19019 T3246 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardRoutingTest-jetty6-1359127503725/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 19075 T3246 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 19147 T3246 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 19248 T3246 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 19254 T3246 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 19257 T3151 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19258 T3151 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"4",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62755_e_vr%2Fg",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62755/e_vr/g"}
[junit4:junit4]   2> 19271 T3169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3245 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19271 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred -

[...truncated too long message...]

 2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  24) Thread[id=1246, name=searcherExecutor-642-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=1179, name=qtp603328428-1179 Acceptor0 SocketConnector@127.0.0.1:14011, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  26) Thread[id=1192, name=searcherExecutor-598-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  27) Thread[id=1176, name=searcherExecutor-588-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {n_f1=PostingsFormat(name=MockSep), foo_b=PostingsFormat(name=TestBloomFilteredLucene41Postings), cat=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), foo_d=PostingsFormat(name=Asserting), foo_f=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tl1=PostingsFormat(name=MockSep), n_d1=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), rnd_b=PostingsFormat(name=Asserting), intDefault=PostingsFormat(name=MockSep), n_td1=PostingsFormat(name=MockSep), timestamp=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), id=PostingsFormat(name=Asserting), range_facet_sl=PostingsFormat(name=Asserting), range_facet_si=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), oddField_s=PostingsFormat(name=Asserting), sequence_i=PostingsFormat(name=Asserting), name=PostingsFormat(name=Asserting), foo_i=PostingsFormat(name=MockSep), regex_dup_B_s=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), multiDefault=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), n_tf1=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), n_dt1=PostingsFormat(name=MockSep), genre_s=PostingsFormat(name=Asserting), author_t=PostingsFormat(name=MockSep), n_ti1=PostingsFormat(name=Asserting), range_facet_l=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), text=PostingsFormat(name=TestBloomFilteredLucene41Postings), _version_=PostingsFormat(name=MockSep), val_i=PostingsFormat(name=Asserting), SubjectTerms_mfacet=PostingsFormat(name=MockSep), series_t=PostingsFormat(name=MockSep), a_t=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), n_tdt1=PostingsFormat(name=MockSep), regex_dup_A_s=PostingsFormat(name=TestBloomFilteredLucene41Postings), price=PostingsFormat(name=TestBloomFilteredLucene41Postings), other_tl1=PostingsFormat(name=MockSep), n_l1=Pulsing41(freqCutoff=10 minBlockSize=88 maxBlockSize=254), a_si=PostingsFormat(name=MockSep), inStock=PostingsFormat(name=TestBloomFilteredLucene41Postings)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=en, timezone=Atlantic/Reykjavik
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=28,free=172784200,total=427819008
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestSurroundQueryParser, ClusterStateUpdateTest, LegacyHTMLStripCharFilterTest, SynonymTokenizerTest, SpellingQueryConverterTest, DOMUtilTest, TermVectorComponentTest, ClusterStateTest, IndexSchemaTest, ConvertedLegacyTest, SolrIndexConfigTest, TimeZoneUtilsTest, StandardRequestHandlerTest, TestReplicationHandler, TestDistributedGrouping, TestDistributedSearch, TestRTGBase, StatsComponentTest, TestUtils, TestPropInject, TestPluginEnable, TestMergePolicyConfig, DirectUpdateHandlerTest, LukeRequestHandlerTest, DebugComponentTest, SignatureUpdateProcessorFactoryTest, TestLRUCache, PingRequestHandlerTest, TestCSVLoader, PathHierarchyTokenizerFactoryTest, FileUtilsTest, TestSolrXMLSerializer, TestFunctionQuery, SuggesterTSTTest, UUIDFieldTest, TestFoldingMultitermQuery, TestSolrDeletionPolicy2, IndexReaderFactoryTest, PrimUtilsTest, TestRemoteStreaming, OverseerCollectionProcessorTest, PeerSyncTest, TestBinaryField, TestDFRSimilarityFactory, TestDocumentBuilder, LeaderElectionTest, MultiTermTest, ZkNodePropsTest, DisMaxRequestHandlerTest, TermVectorComponentDistributedTest, TestPropInjectDefaults, SuggesterTest, DocumentBuilderTest, ReturnFieldsTest, TestWriterPerf, SchemaVersionSpecificBehaviorTest, TestDocSet, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=73DB5873A286C061 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en -Dtests.timezone=Atlantic/Reykjavik -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J1 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([73DB5873A286C061]:0)
[junit4:junit4] Completed on J1 in 7223.69s, 1 test, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 247 suites, 1032 tests, 1 suite-level error, 2 errors, 23 ignored (5 assumptions)

Total time: 145 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3676 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3676/

2 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([AD8248BC50D3AF8D]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([AD8248BC50D3AF8D]:0)




Build Log:
[...truncated 9494 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T362 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /e_li/w
[junit4:junit4]   2> 7 T362 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359100987124
[junit4:junit4]   2> 8 T362 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 21 T363 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 24 T363 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 25 T363 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 25 T363 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 25 T363 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 27 T363 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 122 T362 oasc.ZkTestServer.run start zk server on port:65249
[junit4:junit4]   2> 123 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@209f43e6
[junit4:junit4]   2> 124 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 124 T368 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 125 T368 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 125 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65250
[junit4:junit4]   2> 126 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65250
[junit4:junit4]   2> 126 T366 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 130 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080000 with negotiated timeout 10000 for client /140.211.11.196:65250
[junit4:junit4]   2> 130 T368 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080000, negotiated timeout = 10000
[junit4:junit4]   2> 131 T369 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@209f43e6 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 131 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 132 T362 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 136 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080000
[junit4:junit4]   2> 137 T369 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 138 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65250 which had sessionid 0x13c70bb47080000
[junit4:junit4]   2> 137 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080000 closed
[junit4:junit4]   2> 138 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@44f066ab
[junit4:junit4]   2> 140 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 140 T370 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 141 T370 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 141 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65251
[junit4:junit4]   2> 141 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65251
[junit4:junit4]   2> 143 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080001 with negotiated timeout 10000 for client /140.211.11.196:65251
[junit4:junit4]   2> 143 T370 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080001, negotiated timeout = 10000
[junit4:junit4]   2> 144 T371 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44f066ab name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 144 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 145 T362 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 150 T362 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 159 T362 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 164 T362 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 168 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 170 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 188 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 189 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 321 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 322 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 329 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 330 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 335 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 337 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 343 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 345 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 350 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 351 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 357 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 358 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 363 T362 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 364 T362 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 372 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080001
[junit4:junit4]   2> 373 T371 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 373 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65251 which had sessionid 0x13c70bb47080001
[junit4:junit4]   2> 373 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080001 closed
[junit4:junit4]   2> 524 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 529 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65252
[junit4:junit4]   2> 529 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 530 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 530 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491
[junit4:junit4]   2> 531 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/solr.xml
[junit4:junit4]   2> 531 T362 oasc.CoreContainer.<init> New CoreContainer 581217075
[junit4:junit4]   2> 532 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/'
[junit4:junit4]   2> 533 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/'
[junit4:junit4]   2> 588 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 588 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 589 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 589 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 590 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 590 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 591 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 591 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 592 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 592 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 618 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 638 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 638 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 639 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@46f5c7a4
[junit4:junit4]   2> 640 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 640 T381 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 641 T381 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 642 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65254
[junit4:junit4]   2> 642 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65254
[junit4:junit4]   2> 645 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080002 with negotiated timeout 20000 for client /140.211.11.196:65254
[junit4:junit4]   2> 645 T381 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080002, negotiated timeout = 20000
[junit4:junit4]   2> 646 T382 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46f5c7a4 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 646 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 648 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080002
[junit4:junit4]   2> 649 T382 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 650 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65254 which had sessionid 0x13c70bb47080002
[junit4:junit4]   2> 649 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080002 closed
[junit4:junit4]   2> 651 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 663 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6bf90542
[junit4:junit4]   2> 665 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 665 T383 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 666 T383 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 666 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65255
[junit4:junit4]   2> 667 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65255
[junit4:junit4]   2> 669 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080003 with negotiated timeout 20000 for client /140.211.11.196:65255
[junit4:junit4]   2> 669 T383 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080003, negotiated timeout = 20000
[junit4:junit4]   2> 670 T384 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6bf90542 name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 670 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 673 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 681 T362 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 685 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65252_e_li%2Fw
[junit4:junit4]   2> 686 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:65252_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:65252_e_li%2Fw
[junit4:junit4]   2> 688 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65252_e_li%2Fw
[junit4:junit4]   2> 694 T362 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 710 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 711 T362 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 717 T362 oasc.Overseer.start Overseer (id=89070042293469187-127.0.0.1:65252_e_li%2Fw-n_0000000000) starting
[junit4:junit4]   2> 718 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 720 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 728 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 730 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 736 T386 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 738 T362 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 750 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 758 T385 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 769 T387 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1
[junit4:junit4]   2> 770 T387 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 771 T387 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 771 T387 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 775 T387 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1/'
[junit4:junit4]   2> 777 T387 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1/lib/README' to classloader
[junit4:junit4]   2> 777 T387 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 833 T387 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 930 T387 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1032 T387 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1048 T387 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1763 T387 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1776 T387 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1781 T387 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1805 T387 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1813 T387 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1821 T387 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1824 T387 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1826 T387 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359100987491/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/
[junit4:junit4]   2> 1826 T387 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 1827 T387 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 1862 T387 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data forceNew:false
[junit4:junit4]   2> 1863 T387 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data
[junit4:junit4]   2> 1863 T387 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index/
[junit4:junit4]   2> 1864 T387 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1865 T387 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index forceNew:false
[junit4:junit4]   2> 1873 T387 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a7e9db9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1873 T387 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1874 T387 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index
[junit4:junit4]   2> 1874 T387 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1877 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1877 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1877 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1878 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1879 T387 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1879 T387 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1879 T387 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1880 T387 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1881 T387 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1881 T387 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1882 T387 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1882 T387 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1883 T387 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1883 T387 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1884 T387 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1884 T387 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1885 T387 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1886 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1886 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1887 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1887 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1888 T387 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1889 T387 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1889 T387 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1890 T387 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1891 T387 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1905 T387 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1913 T387 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1915 T387 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1919 T387 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data
[junit4:junit4]   2> 1922 T387 oass.SolrIndexSearcher.<init> Opening Searcher@66ab98b6 main
[junit4:junit4]   2> 1923 T387 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1923 T387 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1924 T387 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1955 T387 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2106 T388 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@66ab98b6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2111 T387 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 2111 T387 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2112 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2265 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2266 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65252_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65252/e_li/w"}
[junit4:junit4]   2> 2267 T385 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2301 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2302 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2327 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3127 T387 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 3127 T387 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65252/e_li/w collection:control_collection shard:shard1
[junit4:junit4]   2> 3143 T387 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 3163 T387 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 3185 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 3186 T387 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 3187 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3191 T387 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 3191 T387 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 3192 T387 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:65252/e_li/w/collection1/
[junit4:junit4]   2> 3192 T387 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 3193 T387 oasc.SyncStrategy.syncToMe http://127.0.0.1:65252/e_li/w/collection1/ has no replicas
[junit4:junit4]   2> 3193 T387 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:65252/e_li/w/collection1/
[junit4:junit4]   2> 3194 T387 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 3201 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3835 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3854 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3919 T387 oasc.ZkController.register We are http://127.0.0.1:65252/e_li/w/collection1/ and leader is http://127.0.0.1:65252/e_li/w/collection1/
[junit4:junit4]   2> 3920 T387 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65252/e_li/w
[junit4:junit4]   2> 3920 T387 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3920 T387 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3921 T387 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3925 T387 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3927 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3928 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3929 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3944 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3963 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@543ff9fb
[junit4:junit4]   2> 3964 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3965 T389 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3966 T389 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 3966 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65258
[junit4:junit4]   2> 3967 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65258
[junit4:junit4]   2> 3968 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080004 with negotiated timeout 10000 for client /140.211.11.196:65258
[junit4:junit4]   2> 3968 T389 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080004, negotiated timeout = 10000
[junit4:junit4]   2> 3969 T390 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@543ff9fb name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3969 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3972 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3979 T362 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4158 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4161 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65259
[junit4:junit4]   2> 4162 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4163 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4163 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095
[junit4:junit4]   2> 4164 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/solr.xml
[junit4:junit4]   2> 4165 T362 oasc.CoreContainer.<init> New CoreContainer 1473632271
[junit4:junit4]   2> 4165 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/'
[junit4:junit4]   2> 4166 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/'
[junit4:junit4]   2> 4225 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4226 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4227 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4227 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4228 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4228 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4229 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4230 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4230 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4231 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4258 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4284 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 4285 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4286 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@508041a6
[junit4:junit4]   2> 4287 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4288 T400 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4289 T400 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 4289 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65260
[junit4:junit4]   2> 4289 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65260
[junit4:junit4]   2> 4292 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080005 with negotiated timeout 20000 for client /140.211.11.196:65260
[junit4:junit4]   2> 4292 T400 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080005, negotiated timeout = 20000
[junit4:junit4]   2> 4292 T401 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@508041a6 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4293 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4295 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080005
[junit4:junit4]   2> 4304 T401 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4304 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080005 closed
[junit4:junit4]   2> 4304 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65260 which had sessionid 0x13c70bb47080005
[junit4:junit4]   2> 4305 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4320 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@16336f1a
[junit4:junit4]   2> 4322 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4322 T402 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4323 T402 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 4323 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65261
[junit4:junit4]   2> 4324 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65261
[junit4:junit4]   2> 4325 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080006 with negotiated timeout 20000 for client /140.211.11.196:65261
[junit4:junit4]   2> 4325 T402 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080006, negotiated timeout = 20000
[junit4:junit4]   2> 4326 T403 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16336f1a name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4326 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4328 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4330 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4337 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65259_e_li%2Fw
[junit4:junit4]   2> 4338 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:65259_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:65259_e_li%2Fw
[junit4:junit4]   2> 4340 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65259_e_li%2Fw
[junit4:junit4]   2> 4343 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4345 T390 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4348 T384 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4352 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4361 T404 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1
[junit4:junit4]   2> 4361 T404 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4362 T404 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4363 T404 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4365 T404 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1/'
[junit4:junit4]   2> 4366 T404 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1/lib/README' to classloader
[junit4:junit4]   2> 4367 T404 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4435 T404 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4530 T404 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4631 T404 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4646 T404 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5312 T404 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5320 T404 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5323 T404 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5339 T404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5343 T404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5347 T404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5348 T404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5349 T404 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359100991095/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/
[junit4:junit4]   2> 5349 T404 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 5350 T404 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 5351 T404 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1 forceNew:false
[junit4:junit4]   2> 5351 T404 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 5351 T404 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index/
[junit4:junit4]   2> 5352 T404 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5353 T404 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index forceNew:false
[junit4:junit4]   2> 5357 T404 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f148d7f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5357 T404 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5358 T404 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index
[junit4:junit4]   2> 5358 T404 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5360 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5360 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5360 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5361 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5361 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5361 T404 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5362 T404 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5362 T404 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5363 T404 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5363 T404 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5364 T404 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5364 T404 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5365 T404 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5365 T404 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5365 T404 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5366 T404 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5366 T404 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5367 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5368 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5368 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5369 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5370 T404 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5370 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5370 T404 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5371 T404 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5371 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65252_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65252/e_li/w"}
[junit4:junit4]   2> 5372 T404 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5372 T404 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5381 T404 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5384 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5384 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5384 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5385 T404 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5387 T404 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5390 T404 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 5391 T404 oass.SolrIndexSearcher.<init> Opening Searcher@45d430ea main
[junit4:junit4]   2> 5392 T404 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5392 T404 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5393 T404 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5398 T404 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5444 T405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@45d430ea main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5449 T404 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5449 T404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6892 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6893 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65259_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65259/e_li/w"}
[junit4:junit4]   2> 6893 T385 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 6894 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6929 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6929 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6929 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7453 T404 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7453 T404 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65259/e_li/w collection:collection1 shard:shard1
[junit4:junit4]   2> 7454 T404 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7477 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7478 T404 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7479 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7489 T404 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7490 T404 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7490 T404 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:65259/e_li/w/collection1/
[junit4:junit4]   2> 7490 T404 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7491 T404 oasc.SyncStrategy.syncToMe http://127.0.0.1:65259/e_li/w/collection1/ has no replicas
[junit4:junit4]   2> 7491 T404 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:65259/e_li/w/collection1/
[junit4:junit4]   2> 7491 T404 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7498 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8436 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8484 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8484 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8484 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8526 T404 oasc.ZkController.register We are http://127.0.0.1:65259/e_li/w/collection1/ and leader is http://127.0.0.1:65259/e_li/w/collection1/
[junit4:junit4]   2> 8526 T404 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65259/e_li/w
[junit4:junit4]   2> 8526 T404 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8527 T404 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8527 T404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8530 T404 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8532 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8532 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8533 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8718 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8721 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65264
[junit4:junit4]   2> 8722 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8722 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8723 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665
[junit4:junit4]   2> 8724 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/solr.xml
[junit4:junit4]   2> 8724 T362 oasc.CoreContainer.<init> New CoreContainer 161107304
[junit4:junit4]   2> 8725 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/'
[junit4:junit4]   2> 8725 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/'
[junit4:junit4]   2> 8781 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 8782 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8782 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8783 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8783 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8784 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8784 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8785 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8785 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8786 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8808 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8828 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 8829 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8829 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3597892
[junit4:junit4]   2> 8830 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8831 T415 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8832 T415 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 8832 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65265
[junit4:junit4]   2> 8833 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65265
[junit4:junit4]   2> 8835 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080007 with negotiated timeout 20000 for client /140.211.11.196:65265
[junit4:junit4]   2> 8835 T415 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080007, negotiated timeout = 20000
[junit4:junit4]   2> 8835 T416 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3597892 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8836 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8837 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080007
[junit4:junit4]   2> 8846 T416 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8846 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65265 which had sessionid 0x13c70bb47080007
[junit4:junit4]   2> 8846 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080007 closed
[junit4:junit4]   2> 8847 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 8859 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6b680257
[junit4:junit4]   2> 8860 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8860 T417 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8861 T417 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 8861 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65266
[junit4:junit4]   2> 8862 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65266
[junit4:junit4]   2> 8863 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080008 with negotiated timeout 20000 for client /140.211.11.196:65266
[junit4:junit4]   2> 8863 T417 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080008, negotiated timeout = 20000
[junit4:junit4]   2> 8864 T418 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b680257 name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8864 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8865 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8867 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8875 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65264_e_li%2Fw
[junit4:junit4]   2> 8875 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:65264_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:65264_e_li%2Fw
[junit4:junit4]   2> 8877 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65264_e_li%2Fw
[junit4:junit4]   2> 8880 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8880 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8880 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8882 T384 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8883 T390 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8884 T403 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8885 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 8893 T419 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1
[junit4:junit4]   2> 8894 T419 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 8895 T419 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 8895 T419 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 8897 T419 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1/'
[junit4:junit4]   2> 8898 T419 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1/lib/README' to classloader
[junit4:junit4]   2> 8899 T419 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 8955 T419 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9028 T419 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9129 T419 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9139 T419 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9748 T419 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9756 T419 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9759 T419 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9776 T419 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9780 T419 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9784 T419 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9786 T419 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9787 T419 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359100995665/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/
[junit4:junit4]   2> 9788 T419 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 9788 T419 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 9789 T419 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2 forceNew:false
[junit4:junit4]   2> 9789 T419 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 9790 T419 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index/
[junit4:junit4]   2> 9790 T419 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9791 T419 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index forceNew:false
[junit4:junit4]   2> 9795 T419 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c66c0a2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9795 T419 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 9796 T419 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index
[junit4:junit4]   2> 9796 T419 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9798 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9798 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9799 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9799 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9800 T419 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9800 T419 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9800 T419 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9801 T419 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9801 T419 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9802 T419 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9802 T419 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9803 T419 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9803 T419 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9803 T419 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9804 T419 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9804 T419 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9805 T419 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9805 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9806 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9806 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9807 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9807 T419 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9808 T419 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9809 T419 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9809 T419 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9810 T419 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9818 T419 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9821 T419 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9823 T419 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 9824 T419 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 9826 T419 oass.SolrIndexSearcher.<init> Opening Searcher@52a63e02 main
[junit4:junit4]   2> 9827 T419 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9827 T419 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9827 T419 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9833 T419 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 9880 T420 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@52a63e02 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 9885 T419 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 9885 T419 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9997 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9998 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65259_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65259/e_li/w"}
[junit4:junit4]   2> 10004 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65264_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65264/e_li/w"}
[junit4:junit4]   2> 10004 T385 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 10004 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 10016 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10016 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10016 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10016 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10889 T419 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 10889 T419 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65264/e_li/w collection:collection1 shard:shard2
[junit4:junit4]   2> 10890 T419 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 10914 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:delete cxid:0x42 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 10915 T419 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 10916 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:create cxid:0x43 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10927 T419 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 10927 T419 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 10928 T419 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:65264/e_li/w/collection1/
[junit4:junit4]   2> 10928 T419 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 10928 T419 oasc.SyncStrategy.syncToMe http://127.0.0.1:65264/e_li/w/collection1/ has no replicas
[junit4:junit4]   2> 10929 T419 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:65264/e_li/w/collection1/
[junit4:junit4]   2> 10929 T419 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 10934 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb47080008 type:create cxid:0x4c zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11531 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11554 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11554 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11554 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11554 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11599 T419 oasc.ZkController.register We are http://127.0.0.1:65264/e_li/w/collection1/ and leader is http://127.0.0.1:65264/e_li/w/collection1/
[junit4:junit4]   2> 11599 T419 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65264/e_li/w
[junit4:junit4]   2> 11599 T419 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 11600 T419 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 11600 T419 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11602 T419 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11603 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11604 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11605 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11793 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 11797 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:33672
[junit4:junit4]   2> 11797 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11798 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11799 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737
[junit4:junit4]   2> 11799 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/solr.xml
[junit4:junit4]   2> 11800 T362 oasc.CoreContainer.<init> New CoreContainer 2124424930
[junit4:junit4]   2> 11800 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/'
[junit4:junit4]   2> 11801 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/'
[junit4:junit4]   2> 11856 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 11857 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11857 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 11858 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11858 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11859 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11859 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11860 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11860 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11861 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 11885 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11905 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 11906 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 11906 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@59f382ac
[junit4:junit4]   2> 11908 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11908 T430 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11909 T430 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 11910 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22663
[junit4:junit4]   2> 11910 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22663
[junit4:junit4]   2> 11923 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb47080009 with negotiated timeout 20000 for client /140.211.11.196:22663
[junit4:junit4]   2> 11923 T430 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb47080009, negotiated timeout = 20000
[junit4:junit4]   2> 11924 T431 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59f382ac name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11924 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11927 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb47080009
[junit4:junit4]   2> 11946 T431 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 11946 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22663 which had sessionid 0x13c70bb47080009
[junit4:junit4]   2> 11946 T362 oaz.ZooKeeper.close Session: 0x13c70bb47080009 closed
[junit4:junit4]   2> 11947 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 11962 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1afddef2
[junit4:junit4]   2> 11963 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11963 T432 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11964 T432 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 11964 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:44735
[junit4:junit4]   2> 11965 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:44735
[junit4:junit4]   2> 11975 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb4708000a with negotiated timeout 20000 for client /140.211.11.196:44735
[junit4:junit4]   2> 11976 T432 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb4708000a, negotiated timeout = 20000
[junit4:junit4]   2> 11976 T433 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1afddef2 name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11977 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11979 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11992 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11999 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33672_e_li%2Fw
[junit4:junit4]   2> 12000 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:33672_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:33672_e_li%2Fw
[junit4:junit4]   2> 12006 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33672_e_li%2Fw
[junit4:junit4]   2> 12010 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12010 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12010 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12010 T418 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12011 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12018 T403 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12018 T384 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12018 T390 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12020 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12029 T434 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1
[junit4:junit4]   2> 12029 T434 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12030 T434 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12030 T434 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12032 T434 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1/'
[junit4:junit4]   2> 12033 T434 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1/lib/README' to classloader
[junit4:junit4]   2> 12034 T434 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12089 T434 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12187 T434 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12288 T434 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12298 T434 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12931 T434 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 12944 T434 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 12959 T434 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12980 T434 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12986 T434 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12991 T434 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12993 T434 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12995 T434 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359100998737/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/
[junit4:junit4]   2> 12995 T434 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 12996 T434 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 12997 T434 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3 forceNew:false
[junit4:junit4]   2> 12997 T434 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 12998 T434 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index/
[junit4:junit4]   2> 12998 T434 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12999 T434 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index forceNew:false
[junit4:junit4]   2> 13042 T434 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@52dd2f52),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13043 T434 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13043 T434 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index
[junit4:junit4]   2> 13044 T434 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13047 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13047 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13047 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13048 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13049 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13049 T434 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13050 T434 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13050 T434 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13051 T434 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13052 T434 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13053 T434 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13053 T434 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13054 T434 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13054 T434 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13055 T434 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13056 T434 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13056 T434 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13057 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13058 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13059 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13060 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13060 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13061 T434 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13062 T434 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13063 T434 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13063 T434 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13068 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13069 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65264_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65264/e_li/w"}
[junit4:junit4]   2> 13076 T434 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13081 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13081 T434 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13085 T434 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13088 T434 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 13090 T434 oass.SolrIndexSearcher.<init> Opening Searcher@23ac1194 main
[junit4:junit4]   2> 13091 T434 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13091 T434 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13092 T434 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13099 T434 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13153 T435 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23ac1194 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13161 T434 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13161 T434 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14587 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14589 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:33672_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:33672/e_li/w"}
[junit4:junit4]   2> 14589 T385 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14589 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 14605 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14606 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14606 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14606 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14606 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15165 T434 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15165 T434 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33672/e_li/w collection:collection1 shard:shard1
[junit4:junit4]   2> 15188 T434 oasc.ZkController.register We are http://127.0.0.1:33672/e_li/w/collection1/ and leader is http://127.0.0.1:65259/e_li/w/collection1/
[junit4:junit4]   2> 15188 T434 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33672/e_li/w
[junit4:junit4]   2> 15189 T434 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15189 T434 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C33 name=collection1 org.apache.solr.core.SolrCore@717a69a8 url=http://127.0.0.1:33672/e_li/w/collection1 node=127.0.0.1:33672_e_li%2Fw C33_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:33672_e_li%2Fw, base_url=http://127.0.0.1:33672/e_li/w}
[junit4:junit4]   2> 15201 T436 C33 P33672 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15202 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15203 T436 C33 P33672 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15203 T436 C33 P33672 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15219 T434 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15221 T436 C33 P33672 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15222 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 15222 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15223 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15446 T362 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15449 T362 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10176
[junit4:junit4]   2> 15450 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15450 T362 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15450 T362 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363
[junit4:junit4]   2> 15451 T362 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/solr.xml
[junit4:junit4]   2> 15451 T362 oasc.CoreContainer.<init> New CoreContainer 2028850196
[junit4:junit4]   2> 15452 T362 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/'
[junit4:junit4]   2> 15452 T362 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/'
[junit4:junit4]   2> 15502 T362 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 15502 T362 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15503 T362 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15503 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15503 T362 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15504 T362 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15504 T362 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15504 T362 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15505 T362 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15505 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15522 T362 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15537 T362 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:65249/solr
[junit4:junit4]   2> 15538 T362 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15538 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4d39f2f6
[junit4:junit4]   2> 15539 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15539 T446 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15540 T446 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 15540 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18886
[junit4:junit4]   2> 15541 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18886
[junit4:junit4]   2> 15550 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb4708000b with negotiated timeout 20000 for client /140.211.11.196:18886
[junit4:junit4]   2> 15550 T446 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb4708000b, negotiated timeout = 20000
[junit4:junit4]   2> 15551 T447 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d39f2f6 name:ZooKeeperConnection Watcher:127.0.0.1:65249 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15551 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15552 T367 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c70bb4708000b
[junit4:junit4]   2> 15556 T447 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15556 T364 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18886 which had sessionid 0x13c70bb4708000b
[junit4:junit4]   2> 15556 T362 oaz.ZooKeeper.close Session: 0x13c70bb4708000b closed
[junit4:junit4]   2> 15557 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 15565 T362 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:65249/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5173890f
[junit4:junit4]   2> 15565 T362 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15566 T448 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:65249. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15566 T448 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:65249, initiating session
[junit4:junit4]   2> 15566 T364 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:29970
[junit4:junit4]   2> 15567 T364 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:29970
[junit4:junit4]   2> 15575 T366 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c70bb4708000c with negotiated timeout 20000 for client /140.211.11.196:29970
[junit4:junit4]   2> 15575 T448 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:65249, sessionid = 0x13c70bb4708000c, negotiated timeout = 20000
[junit4:junit4]   2> 15576 T449 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5173890f name:ZooKeeperConnection Watcher:127.0.0.1:65249/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15576 T362 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15577 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000c type:create cxid:0x1 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15583 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000c type:create cxid:0x2 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15594 T362 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10176_e_li%2Fw
[junit4:junit4]   2> 15595 T367 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c70bb4708000c type:delete cxid:0x4 zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:10176_e_li%2Fw Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:10176_e_li%2Fw
[junit4:junit4]   2> 15596 T362 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10176_e_li%2Fw
[junit4:junit4]   2> 15604 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15604 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15604 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15604 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15605 T418 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15605 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15607 T384 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15607 T403 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15608 T433 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15607 T390 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15609 T362 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15616 T450 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1
[junit4:junit4]   2> 15616 T450 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15617 T450 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15617 T450 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15619 T450 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1/'
[junit4:junit4]   2> 15621 T450 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1/lib/README' to classloader
[junit4:junit4]   2> 15621 T450 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15681 T450 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 15759 T450 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 15861 T450 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 15869 T450 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 16112 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16114 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:33672_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:33672/e_li/w"}
[junit4:junit4]   2> 16224 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16225 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16224 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16224 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16224 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16225 T449 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16544 T450 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16557 T450 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16561 T450 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16582 T450 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16589 T450 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16594 T450 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16596 T450 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16598 T450 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359101002363/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/
[junit4:junit4]   2> 16598 T450 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346421c8
[junit4:junit4]   2> 16599 T450 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 16600 T450 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4 forceNew:false
[junit4:junit4]   2> 16601 T450 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 16601 T450 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index/
[junit4:junit4]   2> 16602 T450 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16603 T450 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index forceNew:false
[junit4:junit4]   2> 16612 T450 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@84efce4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16613 T450 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 16614 T450 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index
[junit4:junit4]   2> 16614 T450 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16617 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16618 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16618 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16618 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16619 T450 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16620 T450 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16620 T450 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16621 T450 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16622 T450 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16623 T450 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16624 T450 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16624 T450 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16625 T450 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16625 T450 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16626 T450 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16627 T450 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16628 T450 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16628 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16629 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16630 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16631 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16631 T450 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16632 T450 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16633 T450 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16634 T450 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16635 T450 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16651 T450 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16657 T450 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16659 T450 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16662 T450 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 16664 T450 oass.SolrIndexSearcher.<init> Opening Searcher@27fa1b48 main
[junit4:junit4]   2> 16665 T450 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16665 T450 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16666 T450 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16675 T450 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 16787 T451 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@27fa1b48 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 16795 T450 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 16795 T450 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17745 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17747 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10176_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10176/e_li/w"}
[junit4:junit4]   2> 17747 T385 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 17747 T385 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 17762 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T449 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17762 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17799 T450 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 17799 T450 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10176/e_li/w collection:collection1 shard:shard2
[junit4:junit4]   2> 17804 T450 oasc.ZkController.register We are http://127.0.0.1:10176/e_li/w/collection1/ and leader is http://127.0.0.1:65264/e_li/w/collection1/
[junit4:junit4]   2> 17804 T450 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10176/e_li/w
[junit4:junit4]   2> 17805 T450 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 17805 T450 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C34 name=collection1 org.apache.solr.core.SolrCore@6f8cf13b url=http://127.0.0.1:10176/e_li/w/collection1 node=127.0.0.1:10176_e_li%2Fw C34_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:10176_e_li%2Fw, base_url=http://127.0.0.1:10176/e_li/w}
[junit4:junit4]   2> 17806 T452 C34 P10176 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 17806 T450 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17807 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 17807 T452 C34 P10176 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 17807 T452 C34 P10176 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17808 T362 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 17809 T362 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 17809 T452 C34 P10176 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17809 T362 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17829 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17832 T362 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 17833 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C33_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:33672_e_li%2Fw, base_url=http://127.0.0.1:33672/e_li/w}
[junit4:junit4]   2> 18278 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:65259/e_li/w/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18289 T436 C33 P33672 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 18309 T436 C33 P33672 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:33672/e_li/w START replicas=[http://127.0.0.1:65259/e_li/w/collection1/] nUpdates=100
[junit4:junit4]   2> 18322 T436 C33 P33672 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18323 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18323 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18323 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18324 T436 C33 P33672 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18324 T436 C33 P33672 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:65259/e_li/w/collection1/. core=collection1
[junit4:junit4]   2> 18324 T436 C33 P33672 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C35 name=collection1 org.apache.solr.core.SolrCore@d3430e url=http://127.0.0.1:65259/e_li/w/collection1 node=127.0.0.1:65259_e_li%2Fw C35_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:65259_e_li%2Fw, base_url=http://127.0.0.1:65259/e_li/w, leader=true}
[junit4:junit4]   2> 18344 T394 C35 P65259 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18348 T395 C35 P65259 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18349 T395 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 18353 T395 C35 P65259 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f148d7f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18354 T395 C35 P65259 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18362 T395 C35 P65259 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f148d7f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f148d7f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18362 T395 C35 P65259 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18364 T395 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 18364 T395 C35 P65259 oass.SolrIndexSearcher.<init> Opening Searcher@20ec3516 realtime
[junit4:junit4]   2> 18365 T395 C35 P65259 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18365 T395 C35 P65259 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4]   2> 18426 T436 C33 P33672 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18427 T436 C33 P33672 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 18455 T436 C33 P33672 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18458 T396 C35 P65259 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18459 T396 C35 P65259 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18460 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18460 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18460 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18464 T396 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 18465 T396 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1/index
[junit4:junit4]   2> 18465 T396 C35 P65259 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18466 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18468 T436 C33 P33672 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 forceNew:false
[junit4:junit4]   2> 18469 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18469 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5) fullCopy=true
[junit4:junit4]   2> 18485 T396 C35 P65259 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=6 
[junit4:junit4]   2> 18492 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index
[junit4:junit4]   2> 18492 T436 C33 P33672 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18495 T436 C33 P33672 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130125100325583
[junit4:junit4]   2> 18497 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18498 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18499 T436 C33 P33672 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583
[junit4:junit4]   2> 18503 T436 C33 P33672 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18503 T436 C33 P33672 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18505 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18510 T436 C33 P33672 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18511 T436 C33 P33672 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18511 T436 C33 P33672 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18512 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583
[junit4:junit4]   2> 18513 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18516 T436 C33 P33672 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18517 T436 C33 P33672 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18517 T436 C33 P33672 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18518 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18518 T436 C33 P33672 oass.SolrIndexSearcher.<init> Opening Searcher@5ac62fa9 main
[junit4:junit4]   2> 18522 T435 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5ac62fa9 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18523 T435 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index
[junit4:junit4]   2> 18523 T436 C33 P33672 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18537 T436 C33 P33672 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c4b75d5),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18538 T436 C33 P33672 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18539 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 18539 T436 C33 P33672 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18540 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index.20130125100325583
[junit4:junit4]   2> 18540 T436 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3/index
[junit4:junit4]   2> 18540 T436 C33 P33672 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18541 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18541 T436 C33 P33672 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18541 T436 C33 P33672 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18543 T436 C33 P33672 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 18836 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19269 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19271 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10176_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10176/e_li/w"}
[junit4:junit4]   2> 19304 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:33672_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:33672/e_li/w"}
[junit4:junit4]   2> 19329 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T449 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19329 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19838 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20842 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C34_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:10176_e_li%2Fw, base_url=http://127.0.0.1:10176/e_li/w}
[junit4:junit4]   2> 21833 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:65264/e_li/w/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 21834 T452 C34 P10176 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:10176/e_li/w START replicas=[http://127.0.0.1:65264/e_li/w/collection1/] nUpdates=100
[junit4:junit4]   2> 21834 T452 C34 P10176 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 21835 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 21836 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 21836 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 21836 T452 C34 P10176 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 21837 T452 C34 P10176 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:65264/e_li/w/collection1/. core=collection1
[junit4:junit4]   2> 21837 T452 C34 P10176 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 21845 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C36 name=collection1 org.apache.solr.core.SolrCore@5373a8a3 url=http://127.0.0.1:65264/e_li/w/collection1 node=127.0.0.1:65264_e_li%2Fw C36_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:65264_e_li%2Fw, base_url=http://127.0.0.1:65264/e_li/w, leader=true}
[junit4:junit4]   2> 21850 T409 C36 P65264 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 21857 T410 C36 P65264 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 21859 T410 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 21864 T410 C36 P65264 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c66c0a2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21865 T410 C36 P65264 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21891 T410 C36 P65264 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c66c0a2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c66c0a2),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21892 T410 C36 P65264 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21893 T410 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 21894 T410 C36 P65264 oass.SolrIndexSearcher.<init> Opening Searcher@47402d88 realtime
[junit4:junit4]   2> 21894 T410 C36 P65264 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21895 T410 C36 P65264 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 38
[junit4:junit4]   2> 21896 T452 C34 P10176 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21897 T452 C34 P10176 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 21900 T411 C36 P65264 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21901 T411 C36 P65264 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 21902 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 21902 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 21902 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 21906 T411 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2> 21906 T411 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2/index
[junit4:junit4]   2> 21907 T411 C36 P65264 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 21908 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 21909 T452 C34 P10176 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 forceNew:false
[junit4:junit4]   2> 21910 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21911 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63) fullCopy=true
[junit4:junit4]   2> 21914 T411 C36 P65264 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 21916 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index
[junit4:junit4]   2> 21917 T452 C34 P10176 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 21935 T452 C34 P10176 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130125100329025
[junit4:junit4]   2> 21936 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21937 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21937 T452 C34 P10176 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025
[junit4:junit4]   2> 21941 T452 C34 P10176 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21942 T452 C34 P10176 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21944 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21944 T452 C34 P10176 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 21945 T452 C34 P10176 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 21945 T452 C34 P10176 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 21946 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025
[junit4:junit4]   2> 21947 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21951 T452 C34 P10176 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21951 T452 C34 P10176 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21952 T452 C34 P10176 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 21952 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21953 T452 C34 P10176 oass.SolrIndexSearcher.<init> Opening Searcher@4bf9418e main
[junit4:junit4]   2> 21959 T451 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4bf9418e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 21960 T451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index
[junit4:junit4]   2> 21960 T452 C34 P10176 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 21992 T452 C34 P10176 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75dd3d63),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 21992 T452 C34 P10176 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 21993 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 21994 T452 C34 P10176 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21994 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index.20130125100329025
[junit4:junit4]   2> 21994 T452 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4/index
[junit4:junit4]   2> 21995 T452 C34 P10176 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 21995 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 21995 T452 C34 P10176 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 21995 T452 C34 P10176 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22020 T452 C34 P10176 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 22347 T385 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22349 T385 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10176_e_li%2Fw",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10176/e_li/w"}
[junit4:junit4]   2> 22359 T384 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22360 T449 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22360 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22360 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22359 T418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22360 T403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22849 T362 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22850 T362 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C37 name=collection1 org.apache.solr.core.SolrCore@36ae2b0 url=http://127.0.0.1:65252/e_li/w/collection1 node=127.0.0.1:65252_e_li%2Fw C37_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:65252_e_li%2Fw, base_url=http://127.0.0.1:65252/e_li/w, leader=true}
[junit4:junit4]   2> 22863 T374 C37 P65252 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data
[junit4:junit4]   2> 22867 T374 C37 P65252 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a7e9db9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 22868 T374 C37 P65252 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 22869 T374 C37 P65252 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/control/data
[junit4:junit4]   2> 22870 T374 C37 P65252 oass.SolrIndexSearcher.<init> Opening Searcher@46ab0975 realtime
[junit4:junit4]   2> 22870 T374 C37 P65252 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425120700638691328)} 0 9
[junit4:junit4]   2> 22888 T397 C35 P65259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty1
[junit4:junit4]   2> 22893 T412 C36 P65264 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty2
[junit4:junit4]   2>  C33_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:33672_e_li%2Fw, base_url=http://127.0.0.1:33672/e_li/w}
[junit4:junit4]   2> 22901 T423 C33 P33672 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty3
[junit4:junit4]   2> 22902 T423 C33 P33672 /update {update.distrib=FROMLEADER&_version_=-1425120700663857152&update.from=http://127.0.0.1:65259/e_li/w/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425120700663857152)} 0 3
[junit4:junit4]   2>  C34_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:10176_e_li%2Fw, base_url=http://127.0.0.1:10176/e_li/w}
[junit4:junit4]   2> 22906 T439 C34 P10176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359100987123/jetty4
[junit4:junit4]   2> 22907 T439 C34 P10176 /update {update.distrib=FROMLEADER&_version_=-1425120700669100032&update.from=http://127.0.0.1:65264/e_li/w/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425120700669100032)} 0 3
[junit4:junit4]   2> 22908 T412 C36 P65264 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425120700669100032)} 0 17
[junit4:junit4]   2> 22909 T397 C35 P65259 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425120700663857152)} 0 23
[junit4:junit4]   2> 22918 T374 C37 P65252 /update {wt=javabin&version=2} {add=[1 (1425120700693217280)]} 0 5
[junit4:junit4]   2> 22939 T423 C33 P33672 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425120700707897344)]} 0 5
[junit4:junit4]   2> 22940 T398 C35 P65259 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1425120700707897344)]} 0 12
[junit4:junit4]   2> 22941 T413 C36 P65264 /update {wt=javabin&version=2} {add=[1]} 0 21
[junit4:junit4]   2> 22947 T374 C37 P65252 /update {wt=javabin&version=2} {add=[2 (1425120700725723136)]} 0 3
[junit4:junit4]   2> 22961 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425120700732014592)]} 0 4
[junit4:junit4]   2> 22962 T413 C36 P65264 /update {wt=javabin&version=2} {add=[2 (1425120700732014592)]} 0 12
[junit4:junit4]   2> 22968 T374 C37 P65252 /update {wt=javabin&version=2} {add=[3 (1425120700747743232)]} 0 2
[junit4:junit4]   2> 22981 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425120700757180416)]} 0 2
[junit4:junit4]   2> 22982 T412 C36 P65264 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1425120700757180416)]} 0 8
[junit4:junit4]   2> 22983 T397 C35 P65259 /update {wt=javabin&version=2} {add=[3]} 0 12
[junit4:junit4]   2> 22989 T374 C37 P65252 /update {wt=javabin&version=2} {add=[4 (1425120700769763328)]} 0 3
[junit4:junit4]   2> 23004 T423 C33 P33672 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425120700779200512)]} 0 3
[junit4:junit4]   2> 23005 T398 C35 P65259 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1425120700779200512)]} 0 9
[junit4:junit4]   2> 23006 T413 C36 P65264 /update {wt=javabin&version=2} {add=[4]} 0 14
[junit4:junit4]   2> 23012 T374 C37 P65252 /update {wt=javabin&version=2} {add=[5 (1425120700793880576)]} 0 3
[junit4:junit4]   2> 23026 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425120700802269184)]} 0 3
[junit4:junit4]   2> 23027 T412 C36 P65264 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1425120700802269184)]} 0 10
[junit4:junit4]   2> 23028 T397 C35 P65259 /update {wt=javabin&version=2} {add=[5]} 0 13
[junit4:junit4]   2> 23034 T374 C37 P65252 /update {wt=javabin&version=2} {add=[6 (1425120700816949248)]} 0 3
[junit4:junit4]   2> 23050 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425120700827435008)]} 0 3
[junit4:junit4]   2> 23051 T412 C36 P65264 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1425120700827435008)]} 0 10
[junit4:junit4]   2> 23052 T397 C35 P65259 /update {wt=javabin&version=2} {add=[6]} 0 15
[junit4:junit4]   2> 23060 T374 C37 P65252 /update {wt=javabin&version=2} {add=[7 (1425120700843163648)]} 0 4
[junit4:junit4]   2> 23072 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425120700851552256)]} 0 3
[junit4:junit4]   2> 23073 T413 C36 P65264 /update {wt=javabin&version=2} {add=[7 (1425120700851552256)]} 0 9
[junit4:junit4]   2> 23080 T374 C37 P65252 /update {wt=javabin&version=2} {add=[8 (1425120700865183744)]} 0 3
[junit4:junit4]   2> 23094 T423 C33 P33672 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425120700871475200)]} 0 4
[junit4:junit4]   2> 23095 T397 C35 P65259 /update {wt=javabin&version=2} {add=[8 (1425120700871475200)]} 0 11
[junit4:junit4]   2> 23102 T374 C37 P65252 /update {wt=javabin&version=2} {add=[9 (1425120700888252416)]} 0 3
[junit4:junit4]   2> 23113 T439 C34 P10176 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425120700894543872)]} 0 3
[junit4:junit4]   2> 23114 T413 C36 P65264 /update {wt=javabin&version=2} {add=[9 (1425120700894543872)]} 0 9
[junit4:junit4]   2> 23138 T374 C37 P65252 /update {wt=javabin&version=2} {add=[10 (1425120700908175360)]} 0 20
[junit4:junit4]   2> 23154 T423 C33 P33672 /update {distrib.from=http://127.0.0.1:65259/e_li/w/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425120700936486912)]} 0 3
[junit4:junit4]   2> 23155 T398 C35 P65259 /update {distrib.from=http://127.0.0.1:65264/e_li/w/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1425120700936486912)]} 0 10
[junit4:junit4]   2> 23156 T413 C36 P

[...truncated too long message...]

ger-9, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  35) Thread[id=507, name=searcherExecutor-287-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  36) Thread[id=542, name=searcherExecutor-309-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  37) Thread[id=372, name=HashSessionScavenger-6, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  38) Thread[id=405, name=searcherExecutor-212-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  39) Thread[id=388, name=searcherExecutor-202-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  40) Thread[id=543, name=searcherExecutor-310-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  41) Thread[id=389, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[AD8248BC50D3AF8D]-SendThread(localhost:65249), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:82)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  42) Thread[id=578, name=searcherExecutor-339-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=en_SG, timezone=Africa/Gaborone
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=45,free=166577424,total=263192576
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SpellPossibilityIteratorTest, SystemInfoHandlerTest, TestQueryTypes, CopyFieldTest, TestLMJelinekMercerSimilarityFactory, CacheHeaderTest, TestElisionMultitermQuery, WordBreakSolrSpellCheckerTest, IndexBasedSpellCheckerTest, TestConfig, SpellCheckCollatorTest, TestQuerySenderNoQuery, TestAtomicUpdateErrorCases, MoreLikeThisHandlerTest, TermsComponentTest, LoggingHandlerTest, TestDefaultSimilarityFactory, TestLFUCache, SolrCmdDistributorTest, FileBasedSpellCheckerTest, AutoCommitTest, FieldAnalysisRequestHandlerTest, CSVRequestHandlerTest, QueryElevationComponentTest, UpdateRequestProcessorFactoryTest, TestXIncludeConfig, ZkControllerTest, TestLuceneMatchVersion, DateFieldTest, SampleTest, FastVectorHighlighterTest, TestWordDelimiterFilterFactory, TestPerFieldSimilarity, DateMathParserTest, PreAnalyzedFieldTest, ChaosMonkeyNothingIsSafeTest, JSONWriterTest, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=AD8248BC50D3AF8D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_SG -Dtests.timezone=Africa/Gaborone -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([AD8248BC50D3AF8D]:0)
[junit4:junit4] Completed on J0 in 7223.98s, 1 test, 2 errors <<< FAILURES!

[...truncated 74 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 247 suites, 1032 tests, 1 suite-level error, 1 error, 66 ignored (5 assumptions)

Total time: 141 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3675 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3675/

2 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([6B4AE0DC02596545]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([6B4AE0DC02596545]:0)




Build Log:
[...truncated 9497 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T1116 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 6 T1116 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359057563823
[junit4:junit4]   2> 7 T1116 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 8 T1117 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 11 T1117 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 11 T1117 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T1117 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 12 T1117 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 13 T1117 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 109 T1116 oasc.ZkTestServer.run start zk server on port:13145
[junit4:junit4]   2> 109 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@8238adc
[junit4:junit4]   2> 110 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 111 T1122 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 111 T1122 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 112 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13146
[junit4:junit4]   2> 112 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13146
[junit4:junit4]   2> 113 T1120 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 116 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60000 with negotiated timeout 10000 for client /140.211.11.196:13146
[junit4:junit4]   2> 116 T1122 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60000, negotiated timeout = 10000
[junit4:junit4]   2> 116 T1123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8238adc name:ZooKeeperConnection Watcher:127.0.0.1:13145 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 117 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 117 T1116 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 121 T1121 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6e24b0b60000
[junit4:junit4]   2> 122 T1123 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 122 T1116 oaz.ZooKeeper.close Session: 0x13c6e24b0b60000 closed
[junit4:junit4]   2> 122 T1118 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13146 which had sessionid 0x13c6e24b0b60000
[junit4:junit4]   2> 123 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@61996d6e
[junit4:junit4]   2> 124 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 124 T1124 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 124 T1124 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 125 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13147
[junit4:junit4]   2> 125 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13147
[junit4:junit4]   2> 126 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60001 with negotiated timeout 10000 for client /140.211.11.196:13147
[junit4:junit4]   2> 127 T1124 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60001, negotiated timeout = 10000
[junit4:junit4]   2> 127 T1125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61996d6e name:ZooKeeperConnection Watcher:127.0.0.1:13145/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 127 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 127 T1116 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 137 T1116 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 140 T1116 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 149 T1116 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 152 T1116 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 153 T1116 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 165 T1116 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 166 T1116 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 271 T1116 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 272 T1116 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 276 T1116 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 276 T1116 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 355 T1116 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 356 T1116 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 359 T1116 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 360 T1116 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 363 T1116 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 364 T1116 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 367 T1116 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 368 T1116 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 371 T1116 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 372 T1116 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 375 T1121 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6e24b0b60001
[junit4:junit4]   2> 382 T1125 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 382 T1118 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13147 which had sessionid 0x13c6e24b0b60001
[junit4:junit4]   2> 382 T1116 oaz.ZooKeeper.close Session: 0x13c6e24b0b60001 closed
[junit4:junit4]   2> 564 T1116 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 567 T1116 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13148
[junit4:junit4]   2> 568 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 568 T1116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 569 T1116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359057564199
[junit4:junit4]   2> 569 T1116 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359057564199/solr.xml
[junit4:junit4]   2> 570 T1116 oasc.CoreContainer.<init> New CoreContainer 787577386
[junit4:junit4]   2> 570 T1116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359057564199/'
[junit4:junit4]   2> 571 T1116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359057564199/'
[junit4:junit4]   2> 632 T1116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 632 T1116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 633 T1116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 633 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 634 T1116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 634 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 635 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 635 T1116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 636 T1116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 636 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 657 T1116 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 677 T1116 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13145/solr
[junit4:junit4]   2> 677 T1116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 678 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2fc24a0f
[junit4:junit4]   2> 679 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 679 T1135 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 680 T1135 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 680 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13149
[junit4:junit4]   2> 681 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13149
[junit4:junit4]   2> 686 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60002 with negotiated timeout 20000 for client /140.211.11.196:13149
[junit4:junit4]   2> 686 T1135 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60002, negotiated timeout = 20000
[junit4:junit4]   2> 686 T1136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fc24a0f name:ZooKeeperConnection Watcher:127.0.0.1:13145 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 686 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 688 T1121 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6e24b0b60002
[junit4:junit4]   2> 689 T1136 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 689 T1118 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13149 which had sessionid 0x13c6e24b0b60002
[junit4:junit4]   2> 689 T1116 oaz.ZooKeeper.close Session: 0x13c6e24b0b60002 closed
[junit4:junit4]   2> 690 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 701 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@511b07b7
[junit4:junit4]   2> 702 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 702 T1137 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 703 T1137 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 703 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13150
[junit4:junit4]   2> 703 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13150
[junit4:junit4]   2> 705 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60003 with negotiated timeout 20000 for client /140.211.11.196:13150
[junit4:junit4]   2> 705 T1137 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60003, negotiated timeout = 20000
[junit4:junit4]   2> 705 T1138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@511b07b7 name:ZooKeeperConnection Watcher:127.0.0.1:13145/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 705 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 707 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 716 T1116 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 718 T1116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13148_
[junit4:junit4]   2> 718 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:13148_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:13148_
[junit4:junit4]   2> 720 T1116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13148_
[junit4:junit4]   2> 730 T1116 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 743 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 744 T1116 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 747 T1116 oasc.Overseer.start Overseer (id=89067196503162883-127.0.0.1:13148_-n_0000000000) starting
[junit4:junit4]   2> 748 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 756 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 757 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 758 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 768 T1140 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 769 T1116 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 772 T1116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 774 T1139 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 779 T1141 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359057564199/collection1
[junit4:junit4]   2> 779 T1141 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 780 T1141 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 780 T1141 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 781 T1141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359057564199/collection1/'
[junit4:junit4]   2> 782 T1141 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359057564199/collection1/lib/README' to classloader
[junit4:junit4]   2> 782 T1141 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359057564199/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 865 T1141 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 971 T1141 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1072 T1141 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1079 T1141 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1770 T1141 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1782 T1141 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1786 T1141 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1801 T1141 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1806 T1141 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1811 T1141 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1812 T1141 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1813 T1141 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359057564199/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data/
[junit4:junit4]   2> 1814 T1141 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@354ec8d6
[junit4:junit4]   2> 1814 T1141 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 1854 T1141 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data forceNew:false
[junit4:junit4]   2> 1855 T1141 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data
[junit4:junit4]   2> 1855 T1141 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data/index/
[junit4:junit4]   2> 1856 T1141 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1857 T1141 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data/index forceNew:false
[junit4:junit4]   2> 1891 T1141 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7752e6a0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1891 T1141 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1892 T1141 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data/index
[junit4:junit4]   2> 1893 T1141 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1894 T1141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1895 T1141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1895 T1141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1895 T1141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1896 T1141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1896 T1141 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1897 T1141 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1897 T1141 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1898 T1141 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1898 T1141 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1898 T1141 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1899 T1141 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1899 T1141 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1899 T1141 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1900 T1141 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1900 T1141 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1901 T1141 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1901 T1141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1902 T1141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1902 T1141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1902 T1141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1903 T1141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1903 T1141 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1904 T1141 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1904 T1141 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1905 T1141 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1921 T1141 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1927 T1141 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1929 T1141 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1931 T1141 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data
[junit4:junit4]   2> 1934 T1141 oass.SolrIndexSearcher.<init> Opening Searcher@9f642ee main
[junit4:junit4]   2> 1935 T1141 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1935 T1141 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1935 T1141 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1954 T1141 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2025 T1142 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9f642ee main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2029 T1141 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 2029 T1141 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2030 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2278 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2278 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13148_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13148"}
[junit4:junit4]   2> 2279 T1139 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2279 T1139 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2279 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2305 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3043 T1141 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 3043 T1141 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:13148 collection:control_collection shard:shard1
[junit4:junit4]   2> 3044 T1141 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 3098 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 3106 T1141 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 3106 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3115 T1141 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 3115 T1141 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 3116 T1141 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:13148/collection1/
[junit4:junit4]   2> 3116 T1141 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 3116 T1141 oasc.SyncStrategy.syncToMe http://127.0.0.1:13148/collection1/ has no replicas
[junit4:junit4]   2> 3116 T1141 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:13148/collection1/
[junit4:junit4]   2> 3116 T1141 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 3132 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3810 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3866 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3908 T1141 oasc.ZkController.register We are http://127.0.0.1:13148/collection1/ and leader is http://127.0.0.1:13148/collection1/
[junit4:junit4]   2> 3908 T1141 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:13148
[junit4:junit4]   2> 3908 T1141 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3908 T1141 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3909 T1141 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3911 T1141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3912 T1116 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3912 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3913 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3923 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3930 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@24c4a893
[junit4:junit4]   2> 3931 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3931 T1143 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3932 T1143 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 3932 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13151
[junit4:junit4]   2> 3933 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13151
[junit4:junit4]   2> 3934 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60004 with negotiated timeout 10000 for client /140.211.11.196:13151
[junit4:junit4]   2> 3934 T1143 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60004, negotiated timeout = 10000
[junit4:junit4]   2> 3934 T1144 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24c4a893 name:ZooKeeperConnection Watcher:127.0.0.1:13145/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3935 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3936 T1116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3938 T1116 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4058 T1116 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4060 T1116 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13152
[junit4:junit4]   2> 4061 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4061 T1116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4062 T1116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359057567754
[junit4:junit4]   2> 4062 T1116 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359057567754/solr.xml
[junit4:junit4]   2> 4062 T1116 oasc.CoreContainer.<init> New CoreContainer 2006658955
[junit4:junit4]   2> 4063 T1116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359057567754/'
[junit4:junit4]   2> 4063 T1116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359057567754/'
[junit4:junit4]   2> 4100 T1116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4100 T1116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4100 T1116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4101 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4101 T1116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4101 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4102 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4102 T1116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4102 T1116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4103 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4116 T1116 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4130 T1116 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13145/solr
[junit4:junit4]   2> 4130 T1116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4131 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2620298d
[junit4:junit4]   2> 4131 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4132 T1154 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4133 T1154 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 4133 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13153
[junit4:junit4]   2> 4133 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13153
[junit4:junit4]   2> 4135 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60005 with negotiated timeout 20000 for client /140.211.11.196:13153
[junit4:junit4]   2> 4135 T1154 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60005, negotiated timeout = 20000
[junit4:junit4]   2> 4135 T1155 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2620298d name:ZooKeeperConnection Watcher:127.0.0.1:13145 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4136 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4137 T1121 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6e24b0b60005
[junit4:junit4]   2> 4138 T1155 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4138 T1118 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13153 which had sessionid 0x13c6e24b0b60005
[junit4:junit4]   2> 4138 T1116 oaz.ZooKeeper.close Session: 0x13c6e24b0b60005 closed
[junit4:junit4]   2> 4139 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4146 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3cf3c6d4
[junit4:junit4]   2> 4147 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4147 T1156 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4148 T1156 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 4148 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13154
[junit4:junit4]   2> 4148 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13154
[junit4:junit4]   2> 4150 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60006 with negotiated timeout 20000 for client /140.211.11.196:13154
[junit4:junit4]   2> 4150 T1156 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60006, negotiated timeout = 20000
[junit4:junit4]   2> 4150 T1157 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cf3c6d4 name:ZooKeeperConnection Watcher:127.0.0.1:13145/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4150 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4151 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4152 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4161 T1116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13152_
[junit4:junit4]   2> 4161 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:13152_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:13152_
[junit4:junit4]   2> 4162 T1116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13152_
[junit4:junit4]   2> 4166 T1144 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4166 T1138 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4166 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 4174 T1116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4180 T1158 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359057567754/collection1
[junit4:junit4]   2> 4180 T1158 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4181 T1158 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4181 T1158 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4183 T1158 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359057567754/collection1/'
[junit4:junit4]   2> 4184 T1158 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359057567754/collection1/lib/README' to classloader
[junit4:junit4]   2> 4185 T1158 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359057567754/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4243 T1158 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4320 T1158 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4420 T1158 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4426 T1158 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5099 T1158 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5111 T1158 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5115 T1158 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5130 T1158 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5135 T1158 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5140 T1158 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5142 T1158 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5143 T1158 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359057567754/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/
[junit4:junit4]   2> 5143 T1158 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@354ec8d6
[junit4:junit4]   2> 5144 T1158 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 5145 T1158 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1 forceNew:false
[junit4:junit4]   2> 5145 T1158 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1
[junit4:junit4]   2> 5146 T1158 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/index/
[junit4:junit4]   2> 5146 T1158 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5147 T1158 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/index forceNew:false
[junit4:junit4]   2> 5156 T1158 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@747127f1),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5157 T1158 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5158 T1158 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/index
[junit4:junit4]   2> 5158 T1158 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5160 T1158 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5160 T1158 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5161 T1158 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5161 T1158 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5162 T1158 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5162 T1158 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5162 T1158 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5163 T1158 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5164 T1158 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5164 T1158 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5165 T1158 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5165 T1158 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5166 T1158 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5166 T1158 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5166 T1158 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5167 T1158 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5168 T1158 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5168 T1158 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5169 T1158 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5169 T1158 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5170 T1158 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5170 T1158 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5171 T1158 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5171 T1158 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5172 T1158 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5173 T1158 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5188 T1158 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5193 T1158 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5195 T1158 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5197 T1158 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1
[junit4:junit4]   2> 5199 T1158 oass.SolrIndexSearcher.<init> Opening Searcher@5296880c main
[junit4:junit4]   2> 5200 T1158 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5201 T1158 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5201 T1158 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5209 T1158 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5265 T1159 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5296880c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5269 T1158 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5269 T1158 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5377 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5378 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13148_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13148"}
[junit4:junit4]   2> 5382 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13152_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13152"}
[junit4:junit4]   2> 5382 T1139 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 5383 T1139 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5395 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5395 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5395 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6272 T1158 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6272 T1158 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:13152 collection:collection1 shard:shard1
[junit4:junit4]   2> 6274 T1158 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6303 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6304 T1158 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6305 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6315 T1158 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6315 T1158 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6315 T1158 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:13152/collection1/
[junit4:junit4]   2> 6315 T1158 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6316 T1158 oasc.SyncStrategy.syncToMe http://127.0.0.1:13152/collection1/ has no replicas
[junit4:junit4]   2> 6316 T1158 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:13152/collection1/
[junit4:junit4]   2> 6316 T1158 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6320 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6911 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6964 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6964 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6964 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6991 T1158 oasc.ZkController.register We are http://127.0.0.1:13152/collection1/ and leader is http://127.0.0.1:13152/collection1/
[junit4:junit4]   2> 6991 T1158 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:13152
[junit4:junit4]   2> 6992 T1158 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6992 T1158 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6992 T1158 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6994 T1158 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6996 T1116 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6997 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6997 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7330 T1116 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7333 T1116 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56495
[junit4:junit4]   2> 7334 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7334 T1116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7335 T1116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359057570836
[junit4:junit4]   2> 7335 T1116 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359057570836/solr.xml
[junit4:junit4]   2> 7336 T1116 oasc.CoreContainer.<init> New CoreContainer 1257919452
[junit4:junit4]   2> 7336 T1116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359057570836/'
[junit4:junit4]   2> 7337 T1116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359057570836/'
[junit4:junit4]   2> 7426 T1116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7427 T1116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7428 T1116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7428 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7429 T1116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7430 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7431 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7432 T1116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7432 T1116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7433 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7462 T1116 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7482 T1116 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13145/solr
[junit4:junit4]   2> 7482 T1116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7483 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@170e308e
[junit4:junit4]   2> 7484 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7484 T1169 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7485 T1169 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 7486 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:59737
[junit4:junit4]   2> 7486 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:59737
[junit4:junit4]   2> 7488 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60007 with negotiated timeout 20000 for client /140.211.11.196:59737
[junit4:junit4]   2> 7488 T1169 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60007, negotiated timeout = 20000
[junit4:junit4]   2> 7488 T1170 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@170e308e name:ZooKeeperConnection Watcher:127.0.0.1:13145 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7489 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7490 T1121 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6e24b0b60007
[junit4:junit4]   2> 7491 T1170 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7491 T1118 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:59737 which had sessionid 0x13c6e24b0b60007
[junit4:junit4]   2> 7491 T1116 oaz.ZooKeeper.close Session: 0x13c6e24b0b60007 closed
[junit4:junit4]   2> 7492 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7503 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@94a5d7b
[junit4:junit4]   2> 7504 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7505 T1171 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7505 T1171 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 7505 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:62424
[junit4:junit4]   2> 7506 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:62424
[junit4:junit4]   2> 7507 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60008 with negotiated timeout 20000 for client /140.211.11.196:62424
[junit4:junit4]   2> 7508 T1171 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60008, negotiated timeout = 20000
[junit4:junit4]   2> 7508 T1172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@94a5d7b name:ZooKeeperConnection Watcher:127.0.0.1:13145/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7508 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7509 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7511 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7512 T1116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56495_
[junit4:junit4]   2> 7513 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:56495_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:56495_
[junit4:junit4]   2> 7522 T1116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56495_
[junit4:junit4]   2> 7524 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7524 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7525 T1138 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7525 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7527 T1157 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7527 T1144 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7528 T1116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7543 T1173 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359057570836/collection1
[junit4:junit4]   2> 7544 T1173 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7546 T1173 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7546 T1173 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7548 T1173 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359057570836/collection1/'
[junit4:junit4]   2> 7550 T1173 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359057570836/collection1/lib/README' to classloader
[junit4:junit4]   2> 7551 T1173 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359057570836/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7635 T1173 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7719 T1173 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7820 T1173 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7827 T1173 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8480 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8481 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13152_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13152"}
[junit4:junit4]   2> 8512 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8512 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8512 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8512 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8516 T1173 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8528 T1173 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8532 T1173 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8547 T1173 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8552 T1173 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8556 T1173 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8558 T1173 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8559 T1173 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359057570836/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/
[junit4:junit4]   2> 8559 T1173 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@354ec8d6
[junit4:junit4]   2> 8560 T1173 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 8561 T1173 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2 forceNew:false
[junit4:junit4]   2> 8561 T1173 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2
[junit4:junit4]   2> 8562 T1173 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/index/
[junit4:junit4]   2> 8563 T1173 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8564 T1173 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/index forceNew:false
[junit4:junit4]   2> 8570 T1173 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@796b45),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8571 T1173 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8571 T1173 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/index
[junit4:junit4]   2> 8572 T1173 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8574 T1173 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8574 T1173 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8575 T1173 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8575 T1173 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8575 T1173 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8576 T1173 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8576 T1173 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8577 T1173 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8577 T1173 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8578 T1173 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8578 T1173 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8579 T1173 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8579 T1173 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8580 T1173 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8580 T1173 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8580 T1173 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8581 T1173 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8581 T1173 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8582 T1173 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8582 T1173 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8583 T1173 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8584 T1173 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8584 T1173 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8585 T1173 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8585 T1173 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8586 T1173 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8600 T1173 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8605 T1173 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8606 T1173 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8609 T1173 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2
[junit4:junit4]   2> 8610 T1173 oass.SolrIndexSearcher.<init> Opening Searcher@64f1c568 main
[junit4:junit4]   2> 8612 T1173 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8612 T1173 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8612 T1173 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8620 T1173 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8675 T1174 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64f1c568 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8680 T1173 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8680 T1173 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10027 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10028 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56495_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56495"}
[junit4:junit4]   2> 10028 T1139 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 10028 T1139 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 10051 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10052 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10051 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10051 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10684 T1173 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 10684 T1173 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56495 collection:collection1 shard:shard2
[junit4:junit4]   2> 10685 T1173 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 10697 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60008 type:delete cxid:0x44 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 10708 T1173 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 10708 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60008 type:create cxid:0x45 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10711 T1173 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 10711 T1173 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 10711 T1173 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56495/collection1/
[junit4:junit4]   2> 10712 T1173 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 10712 T1173 oasc.SyncStrategy.syncToMe http://127.0.0.1:56495/collection1/ has no replicas
[junit4:junit4]   2> 10712 T1173 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56495/collection1/
[junit4:junit4]   2> 10713 T1173 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 10716 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b60008 type:create cxid:0x4e zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11562 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11583 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11583 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11583 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11583 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11591 T1173 oasc.ZkController.register We are http://127.0.0.1:56495/collection1/ and leader is http://127.0.0.1:56495/collection1/
[junit4:junit4]   2> 11591 T1173 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56495
[junit4:junit4]   2> 11591 T1173 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 11591 T1173 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 11591 T1173 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11593 T1173 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11594 T1116 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11595 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11596 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11792 T1116 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 11795 T1116 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:43481
[junit4:junit4]   2> 11795 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11796 T1116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11797 T1116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359057575429
[junit4:junit4]   2> 11797 T1116 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359057575429/solr.xml
[junit4:junit4]   2> 11798 T1116 oasc.CoreContainer.<init> New CoreContainer 1811469248
[junit4:junit4]   2> 11798 T1116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359057575429/'
[junit4:junit4]   2> 11799 T1116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359057575429/'
[junit4:junit4]   2> 11855 T1116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 11855 T1116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11856 T1116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 11856 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11857 T1116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11857 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11858 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11858 T1116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11859 T1116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11859 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 11880 T1116 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11900 T1116 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13145/solr
[junit4:junit4]   2> 11901 T1116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 11901 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4b1e9883
[junit4:junit4]   2> 11902 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11902 T1184 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11903 T1184 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 11904 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22731
[junit4:junit4]   2> 11904 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22731
[junit4:junit4]   2> 11910 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b60009 with negotiated timeout 20000 for client /140.211.11.196:22731
[junit4:junit4]   2> 11910 T1184 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b60009, negotiated timeout = 20000
[junit4:junit4]   2> 11910 T1185 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b1e9883 name:ZooKeeperConnection Watcher:127.0.0.1:13145 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11910 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11912 T1121 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6e24b0b60009
[junit4:junit4]   2> 11923 T1185 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 11923 T1118 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22731 which had sessionid 0x13c6e24b0b60009
[junit4:junit4]   2> 11923 T1116 oaz.ZooKeeper.close Session: 0x13c6e24b0b60009 closed
[junit4:junit4]   2> 11924 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 11935 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@79f1e49a
[junit4:junit4]   2> 11936 T1186 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11936 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11937 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12428
[junit4:junit4]   2> 11937 T1186 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 11937 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12428
[junit4:junit4]   2> 11938 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b6000a with negotiated timeout 20000 for client /140.211.11.196:12428
[junit4:junit4]   2> 11938 T1186 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b6000a, negotiated timeout = 20000
[junit4:junit4]   2> 11938 T1187 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79f1e49a name:ZooKeeperConnection Watcher:127.0.0.1:13145/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11939 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11940 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b6000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11941 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b6000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11949 T1116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43481_
[junit4:junit4]   2> 11949 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b6000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:43481_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:43481_
[junit4:junit4]   2> 11952 T1116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43481_
[junit4:junit4]   2> 11956 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11956 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11956 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11956 T1138 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11957 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11957 T1157 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11957 T1144 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11963 T1172 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11964 T1116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 11970 T1188 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359057575429/collection1
[junit4:junit4]   2> 11970 T1188 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 11971 T1188 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 11971 T1188 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 11973 T1188 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359057575429/collection1/'
[junit4:junit4]   2> 11974 T1188 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359057575429/collection1/lib/README' to classloader
[junit4:junit4]   2> 11975 T1188 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359057575429/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12027 T1188 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12098 T1188 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12199 T1188 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12205 T1188 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12841 T1188 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 12849 T1188 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 12853 T1188 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12865 T1188 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12869 T1188 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12873 T1188 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12874 T1188 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12875 T1188 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359057575429/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/
[junit4:junit4]   2> 12875 T1188 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@354ec8d6
[junit4:junit4]   2> 12875 T1188 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 12876 T1188 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3 forceNew:false
[junit4:junit4]   2> 12876 T1188 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 12877 T1188 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index/
[junit4:junit4]   2> 12877 T1188 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12878 T1188 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index forceNew:false
[junit4:junit4]   2> 12888 T1188 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6a82cd3b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12888 T1188 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 12889 T1188 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index
[junit4:junit4]   2> 12889 T1188 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 12891 T1188 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 12891 T1188 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 12891 T1188 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 12891 T1188 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 12892 T1188 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 12892 T1188 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 12892 T1188 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 12893 T1188 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 12893 T1188 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 12894 T1188 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 12894 T1188 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 12894 T1188 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 12895 T1188 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 12895 T1188 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 12895 T1188 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 12896 T1188 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 12896 T1188 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12896 T1188 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12897 T1188 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12897 T1188 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12898 T1188 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12898 T1188 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12898 T1188 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12899 T1188 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12899 T1188 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 12900 T1188 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 12909 T1188 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 12912 T1188 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 12914 T1188 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 12916 T1188 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 12917 T1188 oass.SolrIndexSearcher.<init> Opening Searcher@6f49194c main
[junit4:junit4]   2> 12918 T1188 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 12918 T1188 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 12919 T1188 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 12924 T1188 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 12975 T1189 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f49194c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12978 T1188 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 12978 T1188 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13095 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13096 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56495_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56495"}
[junit4:junit4]   2> 13101 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:43481_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:43481"}
[junit4:junit4]   2> 13101 T1139 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 13101 T1139 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 13112 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13112 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13112 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13112 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13112 T1187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13981 T1188 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 13981 T1188 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43481 collection:collection1 shard:shard1
[junit4:junit4]   2> 13986 T1188 oasc.ZkController.register We are http://127.0.0.1:43481/collection1/ and leader is http://127.0.0.1:13152/collection1/
[junit4:junit4]   2> 13986 T1188 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43481
[junit4:junit4]   2> 13986 T1188 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 13986 T1188 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C145 name=collection1 org.apache.solr.core.SolrCore@24b01e43 url=http://127.0.0.1:43481/collection1 node=127.0.0.1:43481_ C145_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:43481_, base_url=http://127.0.0.1:43481}
[junit4:junit4]   2> 13987 T1190 C145 P43481 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 13987 T1190 C145 P43481 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 13988 T1190 C145 P43481 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 13988 T1188 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13988 T1190 C145 P43481 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13989 T1190 C145 P43481 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13989 T1116 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 13989 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13990 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14181 T1116 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 14184 T1116 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38451
[junit4:junit4]   2> 14184 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 14185 T1116 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 14185 T1116 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359057577819
[junit4:junit4]   2> 14186 T1116 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359057577819/solr.xml
[junit4:junit4]   2> 14186 T1116 oasc.CoreContainer.<init> New CoreContainer 126095801
[junit4:junit4]   2> 14187 T1116 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359057577819/'
[junit4:junit4]   2> 14187 T1116 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359057577819/'
[junit4:junit4]   2> 14246 T1116 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 14247 T1116 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 14248 T1116 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 14248 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 14249 T1116 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 14249 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 14250 T1116 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 14250 T1116 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 14251 T1116 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 14251 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 14271 T1116 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 14291 T1116 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13145/solr
[junit4:junit4]   2> 14292 T1116 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 14292 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7a6ff61d
[junit4:junit4]   2> 14294 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14294 T1200 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14295 T1200 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 14295 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:33931
[junit4:junit4]   2> 14295 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:33931
[junit4:junit4]   2> 14297 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b6000b with negotiated timeout 20000 for client /140.211.11.196:33931
[junit4:junit4]   2> 14297 T1200 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b6000b, negotiated timeout = 20000
[junit4:junit4]   2> 14297 T1201 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a6ff61d name:ZooKeeperConnection Watcher:127.0.0.1:13145 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14298 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14299 T1121 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c6e24b0b6000b
[junit4:junit4]   2> 14313 T1201 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 14313 T1118 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:33931 which had sessionid 0x13c6e24b0b6000b
[junit4:junit4]   2> 14313 T1116 oaz.ZooKeeper.close Session: 0x13c6e24b0b6000b closed
[junit4:junit4]   2> 14314 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 14320 T1116 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13145/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7001640e
[junit4:junit4]   2> 14321 T1116 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14321 T1202 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13145. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14322 T1202 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13145, initiating session
[junit4:junit4]   2> 14322 T1118 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:56993
[junit4:junit4]   2> 14322 T1118 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:56993
[junit4:junit4]   2> 14324 T1120 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c6e24b0b6000c with negotiated timeout 20000 for client /140.211.11.196:56993
[junit4:junit4]   2> 14324 T1202 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13145, sessionid = 0x13c6e24b0b6000c, negotiated timeout = 20000
[junit4:junit4]   2> 14324 T1203 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7001640e name:ZooKeeperConnection Watcher:127.0.0.1:13145/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14324 T1116 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14325 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b6000c type:create cxid:0x1 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14327 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b6000c type:create cxid:0x2 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14328 T1116 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38451_
[junit4:junit4]   2> 14329 T1121 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c6e24b0b6000c type:delete cxid:0x4 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38451_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38451_
[junit4:junit4]   2> 14337 T1116 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38451_
[junit4:junit4]   2> 14339 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14339 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14339 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14339 T1187 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14339 T1138 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14340 T1187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14340 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14341 T1157 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14341 T1144 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14341 T1172 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14344 T1116 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 14349 T1204 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359057577819/collection1
[junit4:junit4]   2> 14350 T1204 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 14350 T1204 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 14351 T1204 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 14352 T1204 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359057577819/collection1/'
[junit4:junit4]   2> 14354 T1204 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359057577819/collection1/lib/README' to classloader
[junit4:junit4]   2> 14355 T1204 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359057577819/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 14410 T1204 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 14484 T1204 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 14585 T1204 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 14592 T1204 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14625 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14626 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:43481_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:43481"}
[junit4:junit4]   2> 14644 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14644 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14644 T1203 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14644 T1187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14644 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14644 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15057 T1204 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 15065 T1204 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 15068 T1204 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 15079 T1204 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15083 T1204 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15086 T1204 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15087 T1204 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15088 T1204 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359057577819/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/
[junit4:junit4]   2> 15089 T1204 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@354ec8d6
[junit4:junit4]   2> 15089 T1204 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 15090 T1204 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4 forceNew:false
[junit4:junit4]   2> 15090 T1204 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 15090 T1204 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index/
[junit4:junit4]   2> 15091 T1204 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 15091 T1204 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index forceNew:false
[junit4:junit4]   2> 15096 T1204 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@71405088),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15096 T1204 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15097 T1204 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index
[junit4:junit4]   2> 15097 T1204 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 15098 T1204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 15099 T1204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 15099 T1204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 15099 T1204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 15100 T1204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 15100 T1204 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 15100 T1204 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 15100 T1204 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 15101 T1204 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 15101 T1204 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 15102 T1204 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 15102 T1204 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 15102 T1204 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 15102 T1204 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 15103 T1204 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 15103 T1204 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 15104 T1204 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15104 T1204 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15104 T1204 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15105 T1204 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15105 T1204 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15105 T1204 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15106 T1204 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15106 T1204 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15107 T1204 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 15107 T1204 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 15116 T1204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 15119 T1204 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 15120 T1204 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 15122 T1204 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 15123 T1204 oass.SolrIndexSearcher.<init> Opening Searcher@19887741 main
[junit4:junit4]   2> 15124 T1204 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 15124 T1204 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 15124 T1204 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 15137 T1204 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 15201 T1205 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19887741 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 15204 T1204 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15204 T1204 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16149 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16150 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38451_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38451"}
[junit4:junit4]   2> 16151 T1139 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 16151 T1139 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 16171 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16171 T1187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16171 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16171 T1203 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16171 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16171 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16207 T1204 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 16207 T1204 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38451 collection:collection1 shard:shard2
[junit4:junit4]   2> 16212 T1204 oasc.ZkController.register We are http://127.0.0.1:38451/collection1/ and leader is http://127.0.0.1:56495/collection1/
[junit4:junit4]   2> 16212 T1204 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38451
[junit4:junit4]   2> 16212 T1204 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 16212 T1204 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C146 name=collection1 org.apache.solr.core.SolrCore@7795e46 url=http://127.0.0.1:38451/collection1 node=127.0.0.1:38451_ C146_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:38451_, base_url=http://127.0.0.1:38451}
[junit4:junit4]   2> 16214 T1206 C146 P38451 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 16214 T1206 C146 P38451 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 16214 T1204 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16215 T1206 C146 P38451 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 16215 T1206 C146 P38451 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16216 T1116 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 16216 T1206 C146 P38451 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16216 T1116 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 16217 T1116 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16231 T1116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16232 T1116 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 16233 T1116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C145_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:43481_, base_url=http://127.0.0.1:43481}
[junit4:junit4]   2> 17006 T1190 C145 P43481 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:13152/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 17006 T1190 C145 P43481 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43481 START replicas=[http://127.0.0.1:13152/collection1/] nUpdates=100
[junit4:junit4]   2> 17007 T1190 C145 P43481 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 17008 T1190 C145 P43481 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 17008 T1190 C145 P43481 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 17008 T1190 C145 P43481 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 17009 T1190 C145 P43481 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 17009 T1190 C145 P43481 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:13152/collection1/. core=collection1
[junit4:junit4]   2> 17009 T1190 C145 P43481 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C147 name=collection1 org.apache.solr.core.SolrCore@1c2f2e9b url=http://127.0.0.1:13152/collection1 node=127.0.0.1:13152_ C147_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:13152_, base_url=http://127.0.0.1:13152, leader=true}
[junit4:junit4]   2> 17026 T1148 C147 P13152 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 17029 T1149 C147 P13152 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17030 T1149 C147 P13152 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1
[junit4:junit4]   2> 17034 T1149 C147 P13152 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@747127f1),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17035 T1149 C147 P13152 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17064 T1149 C147 P13152 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@747127f1),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@747127f1),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 17065 T1149 C147 P13152 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 17066 T1149 C147 P13152 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1
[junit4:junit4]   2> 17066 T1149 C147 P13152 oass.SolrIndexSearcher.<init> Opening Searcher@504b4119 realtime
[junit4:junit4]   2> 17067 T1149 C147 P13152 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17067 T1149 C147 P13152 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 38
[junit4:junit4]   2> 17068 T1190 C145 P43481 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 17068 T1190 C145 P43481 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 17071 T1150 C147 P13152 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 17071 T1150 C147 P13152 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 17072 T1190 C145 P43481 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 17072 T1190 C145 P43481 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 17072 T1190 C145 P43481 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 17076 T1150 C147 P13152 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1
[junit4:junit4]   2> 17076 T1150 C147 P13152 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1/index
[junit4:junit4]   2> 17077 T1150 C147 P13152 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 17078 T1190 C145 P43481 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 17079 T1190 C145 P43481 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index.20130124155940894 forceNew:false
[junit4:junit4]   2> 17079 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 17080 T1190 C145 P43481 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index.20130124155940894 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2464e88a) fullCopy=true
[junit4:junit4]   2> 17083 T1150 C147 P13152 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 17085 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index
[junit4:junit4]   2> 17085 T1190 C145 P43481 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 17086 T1190 C145 P43481 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130124155940894
[junit4:junit4]   2> 17087 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 17088 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 17088 T1190 C145 P43481 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index.20130124155940894
[junit4:junit4]   2> 17092 T1190 C145 P43481 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index.20130124155940894 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2464e88a),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 17092 T1190 C145 P43481 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 17094 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 17094 T1190 C145 P43481 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 17094 T1190 C145 P43481 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 17095 T1190 C145 P43481 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 17095 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index.20130124155940894
[junit4:junit4]   2> 17096 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 17100 T1190 C145 P43481 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index.20130124155940894 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2464e88a),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 17100 T1190 C145 P43481 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 17100 T1190 C145 P43481 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 17101 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 17101 T1190 C145 P43481 oass.SolrIndexSearcher.<init> Opening Searcher@5b467a6b main
[junit4:junit4]   2> 17103 T1189 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5b467a6b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 17103 T1189 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index
[junit4:junit4]   2> 17104 T1190 C145 P43481 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17106 T1190 C145 P43481 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index.20130124155940894 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2464e88a),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index.20130124155940894 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2464e88a),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17106 T1190 C145 P43481 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17107 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 17108 T1190 C145 P43481 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17108 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index.20130124155940894
[junit4:junit4]   2> 17108 T1190 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3/index
[junit4:junit4]   2> 17108 T1190 C145 P43481 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 17108 T1190 C145 P43481 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 17109 T1190 C145 P43481 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 17109 T1190 C145 P43481 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17117 T1190 C145 P43481 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 17235 T1116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17676 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17677 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38451_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38451"}
[junit4:junit4]   2> 17708 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:43481_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:43481"}
[junit4:junit4]   2> 17724 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17724 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17724 T1203 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17724 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17724 T1187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17724 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18238 T1116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19240 T1116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C146_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:38451_, base_url=http://127.0.0.1:38451}
[junit4:junit4]   2> 20235 T1206 C146 P38451 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:56495/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 20235 T1206 C146 P38451 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38451 START replicas=[http://127.0.0.1:56495/collection1/] nUpdates=100
[junit4:junit4]   2> 20236 T1206 C146 P38451 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20236 T1206 C146 P38451 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20237 T1206 C146 P38451 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20237 T1206 C146 P38451 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 20237 T1206 C146 P38451 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 20237 T1206 C146 P38451 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:56495/collection1/. core=collection1
[junit4:junit4]   2> 20238 T1206 C146 P38451 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20242 T1116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C148 name=collection1 org.apache.solr.core.SolrCore@35f03e02 url=http://127.0.0.1:56495/collection1 node=127.0.0.1:56495_ C148_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56495_, base_url=http://127.0.0.1:56495, leader=true}
[junit4:junit4]   2> 20249 T1163 C148 P56495 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 20254 T1164 C148 P56495 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20255 T1164 C148 P56495 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2
[junit4:junit4]   2> 20259 T1164 C148 P56495 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@796b45),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 20259 T1164 C148 P56495 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 20262 T1164 C148 P56495 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@796b45),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@796b45),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20263 T1164 C148 P56495 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20264 T1164 C148 P56495 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2
[junit4:junit4]   2> 20264 T1164 C148 P56495 oass.SolrIndexSearcher.<init> Opening Searcher@517c8c3c realtime
[junit4:junit4]   2> 20265 T1164 C148 P56495 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20265 T1164 C148 P56495 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4]   2> 20266 T1206 C146 P38451 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20267 T1206 C146 P38451 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 20268 T1165 C148 P56495 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20269 T1165 C148 P56495 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 20270 T1206 C146 P38451 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 20270 T1206 C146 P38451 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 20270 T1206 C146 P38451 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 20273 T1165 C148 P56495 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2
[junit4:junit4]   2> 20274 T1165 C148 P56495 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2/index
[junit4:junit4]   2> 20274 T1165 C148 P56495 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 20275 T1206 C146 P38451 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 20276 T1206 C146 P38451 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index.20130124155944091 forceNew:false
[junit4:junit4]   2> 20277 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 20278 T1206 C146 P38451 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index.20130124155944091 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe0faff) fullCopy=true
[junit4:junit4]   2> 20281 T1165 C148 P56495 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 20283 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index
[junit4:junit4]   2> 20284 T1206 C146 P38451 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 20284 T1206 C146 P38451 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130124155944091
[junit4:junit4]   2> 20285 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 20286 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 20286 T1206 C146 P38451 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index.20130124155944091
[junit4:junit4]   2> 20290 T1206 C146 P38451 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index.20130124155944091 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe0faff),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20290 T1206 C146 P38451 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20292 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 20292 T1206 C146 P38451 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 20293 T1206 C146 P38451 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 20293 T1206 C146 P38451 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 20294 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index.20130124155944091
[junit4:junit4]   2> 20294 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 20298 T1206 C146 P38451 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index.20130124155944091 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe0faff),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20298 T1206 C146 P38451 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20299 T1206 C146 P38451 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 20299 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 20300 T1206 C146 P38451 oass.SolrIndexSearcher.<init> Opening Searcher@4e9bef4b main
[junit4:junit4]   2> 20301 T1205 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e9bef4b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 20302 T1205 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index
[junit4:junit4]   2> 20302 T1206 C146 P38451 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20305 T1206 C146 P38451 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index.20130124155944091 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe0faff),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index.20130124155944091 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe0faff),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 20305 T1206 C146 P38451 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 20306 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 20306 T1206 C146 P38451 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20306 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index.20130124155944091
[junit4:junit4]   2> 20307 T1206 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4/index
[junit4:junit4]   2> 20307 T1206 C146 P38451 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 20307 T1206 C146 P38451 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 20307 T1206 C146 P38451 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 20307 T1206 C146 P38451 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20315 T1206 C146 P38451 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 20742 T1139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20743 T1139 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38451_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38451"}
[junit4:junit4]   2> 20757 T1187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20757 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20757 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20757 T1203 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20757 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20757 T1157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21245 T1116 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21247 T1116 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C149 name=collection1 org.apache.solr.core.SolrCore@5ceb17f3 url=http://127.0.0.1:13148/collection1 node=127.0.0.1:13148_ C149_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:13148_, base_url=http://127.0.0.1:13148, leader=true}
[junit4:junit4]   2> 21262 T1128 C149 P13148 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data
[junit4:junit4]   2> 21267 T1128 C149 P13148 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7752e6a0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21267 T1128 C149 P13148 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21269 T1128 C149 P13148 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/control/data
[junit4:junit4]   2> 21270 T1128 C149 P13148 oass.SolrIndexSearcher.<init> Opening Searcher@3f558376 realtime
[junit4:junit4]   2> 21270 T1128 C149 P13148 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425075166328651776)} 0 10
[junit4:junit4]   2> 21275 T1166 C148 P56495 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty2
[junit4:junit4]   2> 21281 T1151 C147 P13152 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty1
[junit4:junit4]   2>  C146_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:38451_, base_url=http://127.0.0.1:38451}
[junit4:junit4]   2> 21289 T1193 C146 P38451 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty4
[junit4:junit4]   2> 21290 T1193 C146 P38451 /update {update.distrib=FROMLEADER&_version_=-1425075166342283264&update.from=http://127.0.0.1:56495/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425075166342283264)} 0 3
[junit4:junit4]   2>  C145_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43481_, base_url=http://127.0.0.1:43481}
[junit4:junit4]   2> 21294 T1177 C145 P43481 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359057563822/jetty3
[junit4:junit4]   2> 21295 T1177 C145 P43481 /update {update.distrib=FROMLEADER&_version_=-1425075166348574720&update.from=http://127.0.0.1:13152/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425075166348574720)} 0 3
[junit4:junit4]   2> 21295 T1151 C147 P13152 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425075166348574720)} 0 16
[junit4:junit4]   2> 21296 T1166 C148 P56495 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425075166342283264)} 0 23
[junit4:junit4]   2> 21305 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[1 (1425075166371643392)]} 0 5
[junit4:junit4]   2> 21317 T1177 C145 P43481 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425075166377934848)]} 0 4
[junit4:junit4]   2> 21318 T1152 C147 P13152 /update {wt=javabin&version=2} {add=[1 (1425075166377934848)]} 0 11
[junit4:junit4]   2> 21324 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[2 (1425075166393663488)]} 0 3
[junit4:junit4]   2> 21340 T1193 C146 P38451 /update {distrib.from=http://127.0.0.1:56495/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425075166402052096)]} 0 4
[junit4:junit4]   2> 21341 T1167 C148 P56495 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1425075166402052096)]} 0 12
[junit4:junit4]   2> 21342 T1152 C147 P13152 /update {wt=javabin&version=2} {add=[2]} 0 15
[junit4:junit4]   2> 21347 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[3 (1425075166418829312)]} 0 2
[junit4:junit4]   2> 21359 T1193 C146 P38451 /update {distrib.from=http://127.0.0.1:56495/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425075166427217920)]} 0 2
[junit4:junit4]   2> 21360 T1167 C148 P56495 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1425075166427217920)]} 0 7
[junit4:junit4]   2> 21360 T1152 C147 P13152 /update {wt=javabin&version=2} {add=[3]} 0 10
[junit4:junit4]   2> 21366 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[4 (1425075166438752256)]} 0 2
[junit4:junit4]   2> 21375 T1177 C145 P43481 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425075166443995136)]} 0 2
[junit4:junit4]   2> 21376 T1152 C147 P13152 /update {wt=javabin&version=2} {add=[4 (1425075166443995136)]} 0 7
[junit4:junit4]   2> 21381 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[5 (1425075166454480896)]} 0 2
[junit4:junit4]   2> 21393 T1193 C146 P38451 /update {distrib.from=http://127.0.0.1:56495/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425075166462869504)]} 0 2
[junit4:junit4]   2> 21394 T1167 C148 P56495 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1425075166462869504)]} 0 7
[junit4:junit4]   2> 21394 T1152 C147 P13152 /update {wt=javabin&version=2} {add=[5]} 0 10
[junit4:junit4]   2> 21399 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[6 (1425075166473355264)]} 0 2
[junit4:junit4]   2> 21408 T1193 C146 P38451 /update {distrib.from=http://127.0.0.1:56495/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425075166478598144)]} 0 2
[junit4:junit4]   2> 21409 T1166 C148 P56495 /update {wt=javabin&version=2} {add=[6 (1425075166478598144)]} 0 7
[junit4:junit4]   2> 21414 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[7 (1425075166489083904)]} 0 2
[junit4:junit4]   2> 21423 T1193 C146 P38451 /update {distrib.from=http://127.0.0.1:56495/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425075166494326784)]} 0 2
[junit4:junit4]   2> 21424 T1166 C148 P56495 /update {wt=javabin&version=2} {add=[7 (1425075166494326784)]} 0 7
[junit4:junit4]   2> 21429 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[8 (1425075166504812544)]} 0 2
[junit4:junit4]   2> 21439 T1177 C145 P43481 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425075166510055424)]} 0 2
[junit4:junit4]   2> 21440 T1152 C147 P13152 /update {wt=javabin&version=2} {add=[8 (1425075166510055424)]} 0 8
[junit4:junit4]   2> 21445 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[9 (1425075166521589760)]} 0 2
[junit4:junit4]   2> 21457 T1193 C146 P38451 /update {distrib.from=http://127.0.0.1:56495/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425075166529978368)]} 0 2
[junit4:junit4]   2> 21458 T1167 C148 P56495 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1425075166529978368)]} 0 7
[junit4:junit4]   2> 21458 T1152 C147 P13152 /update {wt=javabin&version=2} {add=[9]} 0 10
[junit4:junit4]   2> 21481 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[10 (1425075166540464128)]} 0 20
[junit4:junit4]   2> 21492 T1177 C145 P43481 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425075166564581376)]} 0 2
[junit4:junit4]   2> 21493 T1152 C147 P13152 /update {wt=javabin&version=2} {add=[10 (1425075166564581376)]} 0 8
[junit4:junit4]   2> 21500 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[11 (1425075166577164288)]} 0 4
[junit4:junit4]   2> 21510 T1177 C145 P43481 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1425075166584504320)]} 0 2
[junit4:junit4]   2> 21511 T1152 C147 P13152 /update {wt=javabin&version=2} {add=[11 (1425075166584504320)]} 0 8
[junit4:junit4]   2> 21516 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[12 (1425075166596038656)]} 0 2
[junit4:junit4]   2> 21529 T1177 C145 P43481 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1425075166604427264)]} 0 2
[junit4:junit4]   2> 21530 T1151 C147 P13152 /update {distrib.from=http://127.0.0.1:56495/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1425075166604427264)]} 0 8
[junit4:junit4]   2> 21531 T1166 C148 P56495 /update {wt=javabin&version=2} {add=[12]} 0 12
[junit4:junit4]   2> 21536 T1128 C149 P13148 /update {wt=javabin&version=2} {add=[13 (1425075166617010176)]} 0 2
[junit4:junit4]   2> 21549 T1177 C145 P43481 /update {distrib.from=http://127.0.0.1:13152/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1425075166625398784)]} 0 2
[junit4:junit4]   2> 21550 T1151 C147 P13152 /update {distrib.from=http://127.0.0.1:56495/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1425075166625398784)]} 0 8
[junit4:junit4]   2> 21551 T1166 C148 P5

[...truncated too long message...]

t.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  26) Thread[id=1248, name=qtp1536818551-1248, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.ByteArrayBuffer.readFrom(ByteArrayBuffer.java:375)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.bio.StreamEndPoint.fill(StreamEndPoint.java:141)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.fill(SocketConnector.java:227)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.fill(HttpParser.java:1035)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:280)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  27) Thread[id=1160, name=HashSessionScavenger-18, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  28) Thread[id=1258, name=qtp1830519698-1258, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.ByteArrayBuffer.readFrom(ByteArrayBuffer.java:375)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.bio.StreamEndPoint.fill(StreamEndPoint.java:141)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.fill(SocketConnector.java:227)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.fill(HttpParser.java:1035)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:280)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  29) Thread[id=1215, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[6B4AE0DC02596545]-SendThread(localhost:13145), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.IOUtil.drain(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:167)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:108)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {n_f1=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188), foo_b=PostingsFormat(name=SimpleText), foo_d=Lucene41(blocksize=128), foo_f=PostingsFormat(name=SimpleText), n_tl1=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188), n_d1=Pulsing41(freqCutoff=1 minBlockSize=69 maxBlockSize=188), rnd_b=Lucene41(blocksize=128), intDefault=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188), n_td1=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188), timestamp=Pulsing41(freqCutoff=1 minBlockSize=69 maxBlockSize=188), id=Lucene41(blocksize=128), range_facet_sl=Lucene41(blocksize=128), range_facet_si=Pulsing41(freqCutoff=1 minBlockSize=69 maxBlockSize=188), oddField_s=Lucene41(blocksize=128), multiDefault=Pulsing41(freqCutoff=1 minBlockSize=69 maxBlockSize=188), n_tf1=Pulsing41(freqCutoff=1 minBlockSize=69 maxBlockSize=188), n_dt1=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188), n_ti1=Lucene41(blocksize=128), range_facet_l=Pulsing41(freqCutoff=1 minBlockSize=69 maxBlockSize=188), text=PostingsFormat(name=SimpleText), _version_=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188), SubjectTerms_mfacet=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188), a_t=Pulsing41(freqCutoff=1 minBlockSize=69 maxBlockSize=188), n_tdt1=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188), other_tl1=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188), n_l1=Pulsing41(freqCutoff=1 minBlockSize=69 maxBlockSize=188), a_si=Pulsing41(freqCutoff=20 minBlockSize=69 maxBlockSize=188)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=en_ZA, timezone=America/Curacao
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=40,free=298607408,total=503316480
[junit4:junit4]   2> NOTE: All tests run in this JVM: [CircularListTest, TestSolr4Spatial, TestRandomFaceting, PreAnalyzedFieldTest, TestPseudoReturnFields, UpdateRequestProcessorFactoryTest, ChaosMonkeySafeLeaderTest, TestFuzzyAnalyzedSuggestions, CoreAdminHandlerTest, TestReload, JsonLoaderTest, CacheHeaderTest, TestFaceting, HighlighterConfigTest, SOLR749Test, TestSuggestSpellingConverter, TestDefaultSimilarityFactory, DateFieldTest, SolrPluginUtilsTest, TestElisionMultitermQuery, MBeansHandlerTest, ZkControllerTest, DistributedSpellCheckComponentTest, UniqFieldsUpdateProcessorFactoryTest, TestFiltering, CurrencyFieldTest, TestNumberUtils, DirectSolrConnectionTest, QueryElevationComponentTest, TestRangeQuery, TestJmxMonitoredMap, TestQuerySenderNoQuery, CSVRequestHandlerTest, JSONWriterTest, NumericFieldsTest, TestGroupingSearch, AlternateDirectoryTest, FileBasedSpellCheckerTest, TestLFUCache, TestBadConfig, DistanceFunctionTest, TermsComponentTest, MinimalSchemaTest, CopyFieldTest, TestStressVersions, SortByFunctionTest, CollectionsAPIDistributedZkTest, TestQueryTypes, SpatialFilterTest, NoCacheHeaderTest, TestSearchPerf, TestIndexingPerformance, TestPartialUpdateDeduplication, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, ResourceLoaderTest, FastVectorHighlighterTest, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=6B4AE0DC02596545 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_ZA -Dtests.timezone=America/Curacao -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([6B4AE0DC02596545]:0)
[junit4:junit4] Completed on J0 in 7223.87s, 1 test, 2 errors <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 247 suites, 1032 tests, 1 suite-level error, 1 error, 23 ignored (5 assumptions)

Total time: 146 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure