You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/02/06 13:36:16 UTC

[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3713 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3713/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([F7C2F0E9117976D6]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([F7C2F0E9117976D6]:0)




Build Log:
[...truncated 9549 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T833 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /pun/z
[junit4:junit4]   2> 4 T833 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1360146932413
[junit4:junit4]   2> 5 T833 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 5 T834 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 8 T834 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 9 T834 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 9 T834 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 9 T834 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 10 T834 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 106 T833 oasc.ZkTestServer.run start zk server on port:54984
[junit4:junit4]   2> 106 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4a0a828c
[junit4:junit4]   2> 107 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 107 T839 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 108 T839 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 108 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:41470
[junit4:junit4]   2> 109 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:41470
[junit4:junit4]   2> 109 T837 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 111 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40000 with negotiated timeout 10000 for client /140.211.11.196:41470
[junit4:junit4]   2> 112 T839 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40000, negotiated timeout = 10000
[junit4:junit4]   2> 112 T840 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a0a828c name:ZooKeeperConnection Watcher:127.0.0.1:54984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 112 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 112 T833 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 115 T838 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caf1322c40000
[junit4:junit4]   2> 124 T840 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 124 T833 oaz.ZooKeeper.close Session: 0x13caf1322c40000 closed
[junit4:junit4]   2> 125 T835 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:41470 which had sessionid 0x13caf1322c40000
[junit4:junit4]   2> 125 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@44599ffd
[junit4:junit4]   2> 126 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 126 T841 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 127 T841 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 127 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:55053
[junit4:junit4]   2> 127 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:55053
[junit4:junit4]   2> 128 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40001 with negotiated timeout 10000 for client /140.211.11.196:55053
[junit4:junit4]   2> 128 T841 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40001, negotiated timeout = 10000
[junit4:junit4]   2> 129 T842 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44599ffd name:ZooKeeperConnection Watcher:127.0.0.1:54984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 129 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 129 T833 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 134 T833 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 141 T833 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 145 T833 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 148 T833 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 149 T833 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 162 T833 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 162 T833 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 266 T833 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 267 T833 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 270 T833 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 271 T833 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 281 T833 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 281 T833 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 285 T833 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 285 T833 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 288 T833 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 289 T833 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 292 T833 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 293 T833 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 296 T833 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 296 T833 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 299 T838 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caf1322c40001
[junit4:junit4]   2> 300 T842 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 300 T833 oaz.ZooKeeper.close Session: 0x13caf1322c40001 closed
[junit4:junit4]   2> 301 T835 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:55053 which had sessionid 0x13caf1322c40001
[junit4:junit4]   2> 413 T833 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 417 T833 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62803
[junit4:junit4]   2> 417 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 417 T833 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 418 T833 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360146932710
[junit4:junit4]   2> 418 T833 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360146932710/solr.xml
[junit4:junit4]   2> 418 T833 oasc.CoreContainer.<init> New CoreContainer 673443003
[junit4:junit4]   2> 419 T833 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360146932710/'
[junit4:junit4]   2> 419 T833 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360146932710/'
[junit4:junit4]   2> 455 T833 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 455 T833 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 456 T833 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 456 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 456 T833 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 457 T833 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 457 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 457 T833 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 457 T833 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 458 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 473 T833 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 487 T833 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54984/solr
[junit4:junit4]   2> 487 T833 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 488 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@14b3de01
[junit4:junit4]   2> 488 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 489 T852 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 489 T852 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 490 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:30472
[junit4:junit4]   2> 490 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:30472
[junit4:junit4]   2> 492 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40002 with negotiated timeout 20000 for client /140.211.11.196:30472
[junit4:junit4]   2> 492 T852 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40002, negotiated timeout = 20000
[junit4:junit4]   2> 492 T853 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14b3de01 name:ZooKeeperConnection Watcher:127.0.0.1:54984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 492 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 493 T838 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caf1322c40002
[junit4:junit4]   2> 494 T853 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 494 T835 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:30472 which had sessionid 0x13caf1322c40002
[junit4:junit4]   2> 494 T833 oaz.ZooKeeper.close Session: 0x13caf1322c40002 closed
[junit4:junit4]   2> 495 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 504 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5cfdc8ce
[junit4:junit4]   2> 504 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 505 T854 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 505 T854 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 505 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31480
[junit4:junit4]   2> 506 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31480
[junit4:junit4]   2> 507 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40003 with negotiated timeout 20000 for client /140.211.11.196:31480
[junit4:junit4]   2> 507 T854 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40003, negotiated timeout = 20000
[junit4:junit4]   2> 507 T855 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cfdc8ce name:ZooKeeperConnection Watcher:127.0.0.1:54984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 507 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 509 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 518 T833 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 520 T833 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62803_pun%2Fz
[junit4:junit4]   2> 521 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:62803_pun%2Fz Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:62803_pun%2Fz
[junit4:junit4]   2> 522 T833 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62803_pun%2Fz
[junit4:junit4]   2> 533 T833 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 544 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 546 T833 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 549 T833 oasc.Overseer.start Overseer (id=89138589363077123-127.0.0.1:62803_pun%2Fz-n_0000000000) starting
[junit4:junit4]   2> 550 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 558 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 559 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 561 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 570 T857 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 571 T833 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 581 T833 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 584 T856 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 587 T858 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360146932710/collection1
[junit4:junit4]   2> 587 T858 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 588 T858 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 588 T858 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 590 T858 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360146932710/collection1/'
[junit4:junit4]   2> 591 T858 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360146932710/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 591 T858 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360146932710/collection1/lib/README' to classloader
[junit4:junit4]   2> 644 T858 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 715 T858 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 816 T858 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 822 T858 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1426 T858 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1434 T858 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1436 T858 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1450 T858 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1454 T858 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1457 T858 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1458 T858 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1460 T858 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360146932710/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data/
[junit4:junit4]   2> 1460 T858 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ee76fcc
[junit4:junit4]   2> 1460 T858 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1461 T858 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data forceNew:false
[junit4:junit4]   2> 1462 T858 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data
[junit4:junit4]   2> 1462 T858 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data/index/
[junit4:junit4]   2> 1462 T858 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1463 T858 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data/index forceNew:false
[junit4:junit4]   2> 1468 T858 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6938af32; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1468 T858 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1469 T858 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data/index
[junit4:junit4]   2> 1469 T858 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1471 T858 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1471 T858 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1471 T858 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1471 T858 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1472 T858 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1472 T858 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1472 T858 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1472 T858 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1473 T858 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1473 T858 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1473 T858 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1474 T858 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1474 T858 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1474 T858 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1474 T858 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1475 T858 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1475 T858 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1475 T858 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1476 T858 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1476 T858 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1476 T858 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1477 T858 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1477 T858 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1477 T858 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1478 T858 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1478 T858 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1486 T858 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1490 T858 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1491 T858 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1492 T858 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data
[junit4:junit4]   2> 1494 T858 oass.SolrIndexSearcher.<init> Opening Searcher@c20a695 main
[junit4:junit4]   2> 1494 T858 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1495 T858 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1495 T858 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1511 T858 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1563 T859 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c20a695 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1568 T858 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1568 T858 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1569 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2088 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2089 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62803_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62803/pun/z"}
[junit4:junit4]   2> 2089 T856 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2095 T856 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2095 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2148 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2578 T858 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2578 T858 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62803/pun/z collection:control_collection shard:shard1
[junit4:junit4]   2> 2579 T858 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2602 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2603 T858 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2603 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2617 T858 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2617 T858 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2618 T858 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:62803/pun/z/collection1/
[junit4:junit4]   2> 2618 T858 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2619 T858 oasc.SyncStrategy.syncToMe http://127.0.0.1:62803/pun/z/collection1/ has no replicas
[junit4:junit4]   2> 2619 T858 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:62803/pun/z/collection1/
[junit4:junit4]   2> 2619 T858 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2625 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3660 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3682 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3706 T858 oasc.ZkController.register We are http://127.0.0.1:62803/pun/z/collection1/ and leader is http://127.0.0.1:62803/pun/z/collection1/
[junit4:junit4]   2> 3706 T858 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62803/pun/z
[junit4:junit4]   2> 3706 T858 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3706 T858 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3707 T858 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3709 T858 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3711 T833 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3711 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3712 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3728 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3744 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7a91d22f
[junit4:junit4]   2> 3745 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3746 T860 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3746 T860 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 3746 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:53791
[junit4:junit4]   2> 3747 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:53791
[junit4:junit4]   2> 3748 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40004 with negotiated timeout 10000 for client /140.211.11.196:53791
[junit4:junit4]   2> 3748 T860 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40004, negotiated timeout = 10000
[junit4:junit4]   2> 3749 T861 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a91d22f name:ZooKeeperConnection Watcher:127.0.0.1:54984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3749 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3750 T833 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3754 T833 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3922 T833 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3925 T833 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50938
[junit4:junit4]   2> 3926 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3926 T833 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3927 T833 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360146936163
[junit4:junit4]   2> 3927 T833 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360146936163/solr.xml
[junit4:junit4]   2> 3928 T833 oasc.CoreContainer.<init> New CoreContainer 1648445560
[junit4:junit4]   2> 3929 T833 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360146936163/'
[junit4:junit4]   2> 3929 T833 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360146936163/'
[junit4:junit4]   2> 3984 T833 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 3985 T833 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3985 T833 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 3986 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3986 T833 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3987 T833 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3987 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3988 T833 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3988 T833 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3988 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4010 T833 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4030 T833 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54984/solr
[junit4:junit4]   2> 4030 T833 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4031 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@709dea70
[junit4:junit4]   2> 4032 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4032 T871 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4033 T871 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 4033 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31529
[junit4:junit4]   2> 4034 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31529
[junit4:junit4]   2> 4036 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40005 with negotiated timeout 20000 for client /140.211.11.196:31529
[junit4:junit4]   2> 4036 T871 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40005, negotiated timeout = 20000
[junit4:junit4]   2> 4036 T872 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@709dea70 name:ZooKeeperConnection Watcher:127.0.0.1:54984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4036 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4038 T838 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caf1322c40005
[junit4:junit4]   2> 4048 T872 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4048 T835 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:31529 which had sessionid 0x13caf1322c40005
[junit4:junit4]   2> 4048 T833 oaz.ZooKeeper.close Session: 0x13caf1322c40005 closed
[junit4:junit4]   2> 4049 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4061 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@101fd222
[junit4:junit4]   2> 4062 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4062 T873 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4063 T873 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 4063 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22584
[junit4:junit4]   2> 4063 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22584
[junit4:junit4]   2> 4065 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40006 with negotiated timeout 20000 for client /140.211.11.196:22584
[junit4:junit4]   2> 4065 T873 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40006, negotiated timeout = 20000
[junit4:junit4]   2> 4065 T874 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@101fd222 name:ZooKeeperConnection Watcher:127.0.0.1:54984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4065 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4066 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4072 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4074 T833 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50938_pun%2Fz
[junit4:junit4]   2> 4075 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:50938_pun%2Fz Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:50938_pun%2Fz
[junit4:junit4]   2> 4076 T833 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50938_pun%2Fz
[junit4:junit4]   2> 4084 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4085 T861 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4086 T855 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4089 T833 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4096 T875 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360146936163/collection1
[junit4:junit4]   2> 4096 T875 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4097 T875 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4097 T875 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4098 T875 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360146936163/collection1/'
[junit4:junit4]   2> 4099 T875 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360146936163/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4100 T875 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360146936163/collection1/lib/README' to classloader
[junit4:junit4]   2> 4152 T875 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4224 T875 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4325 T875 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4331 T875 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4924 T875 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4933 T875 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4935 T875 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4949 T875 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4953 T875 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4956 T875 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4958 T875 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4959 T875 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360146936163/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/
[junit4:junit4]   2> 4959 T875 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ee76fcc
[junit4:junit4]   2> 4959 T875 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 4960 T875 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1 forceNew:false
[junit4:junit4]   2> 4960 T875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1
[junit4:junit4]   2> 4961 T875 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/index/
[junit4:junit4]   2> 4961 T875 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4962 T875 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/index forceNew:false
[junit4:junit4]   2> 4966 T875 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e187eda; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4966 T875 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 4967 T875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/index
[junit4:junit4]   2> 4967 T875 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4969 T875 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4969 T875 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4969 T875 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4970 T875 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4970 T875 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4970 T875 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4971 T875 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4971 T875 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4971 T875 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4972 T875 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4972 T875 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4972 T875 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4973 T875 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4973 T875 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4973 T875 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4974 T875 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4974 T875 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4974 T875 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4975 T875 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4975 T875 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4975 T875 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4976 T875 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4976 T875 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4977 T875 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4977 T875 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4977 T875 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 4986 T875 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4989 T875 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 4990 T875 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 4991 T875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1
[junit4:junit4]   2> 4993 T875 oass.SolrIndexSearcher.<init> Opening Searcher@13838b36 main
[junit4:junit4]   2> 4993 T875 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4994 T875 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4994 T875 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 4999 T875 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5045 T876 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13838b36 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5049 T875 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5049 T875 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5189 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5189 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62803_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62803/pun/z"}
[junit4:junit4]   2> 5202 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50938_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50938/pun/z"}
[junit4:junit4]   2> 5202 T856 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 5203 T856 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5213 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5213 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5213 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6051 T875 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6051 T875 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50938/pun/z collection:collection1 shard:shard1
[junit4:junit4]   2> 6052 T875 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6070 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6071 T875 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6071 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6073 T875 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6074 T875 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6074 T875 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50938/pun/z/collection1/
[junit4:junit4]   2> 6074 T875 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6074 T875 oasc.SyncStrategy.syncToMe http://127.0.0.1:50938/pun/z/collection1/ has no replicas
[junit4:junit4]   2> 6075 T875 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50938/pun/z/collection1/
[junit4:junit4]   2> 6075 T875 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6079 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6728 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6752 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6752 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6752 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6796 T875 oasc.ZkController.register We are http://127.0.0.1:50938/pun/z/collection1/ and leader is http://127.0.0.1:50938/pun/z/collection1/
[junit4:junit4]   2> 6796 T875 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50938/pun/z
[junit4:junit4]   2> 6796 T875 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6796 T875 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6797 T875 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6798 T875 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6799 T833 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6800 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6801 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 6987 T833 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 6990 T833 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:17765
[junit4:junit4]   2> 6991 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 6991 T833 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6992 T833 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360146939225
[junit4:junit4]   2> 6992 T833 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360146939225/solr.xml
[junit4:junit4]   2> 6993 T833 oasc.CoreContainer.<init> New CoreContainer 2012997630
[junit4:junit4]   2> 6993 T833 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360146939225/'
[junit4:junit4]   2> 6994 T833 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360146939225/'
[junit4:junit4]   2> 7051 T833 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7051 T833 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7052 T833 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7052 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7053 T833 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7053 T833 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7053 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7054 T833 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7054 T833 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7055 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7077 T833 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7097 T833 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54984/solr
[junit4:junit4]   2> 7098 T833 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7098 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4c965acc
[junit4:junit4]   2> 7099 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7100 T886 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7101 T886 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 7101 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:42923
[junit4:junit4]   2> 7101 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:42923
[junit4:junit4]   2> 7103 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40007 with negotiated timeout 20000 for client /140.211.11.196:42923
[junit4:junit4]   2> 7103 T886 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40007, negotiated timeout = 20000
[junit4:junit4]   2> 7104 T887 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c965acc name:ZooKeeperConnection Watcher:127.0.0.1:54984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7104 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7105 T838 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caf1322c40007
[junit4:junit4]   2> 7150 T887 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7150 T835 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:42923 which had sessionid 0x13caf1322c40007
[junit4:junit4]   2> 7150 T833 oaz.ZooKeeper.close Session: 0x13caf1322c40007 closed
[junit4:junit4]   2> 7151 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7163 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1dbe9cb6
[junit4:junit4]   2> 7164 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7165 T888 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7165 T888 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 7165 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:46360
[junit4:junit4]   2> 7166 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:46360
[junit4:junit4]   2> 7168 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40008 with negotiated timeout 20000 for client /140.211.11.196:46360
[junit4:junit4]   2> 7168 T888 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40008, negotiated timeout = 20000
[junit4:junit4]   2> 7168 T889 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dbe9cb6 name:ZooKeeperConnection Watcher:127.0.0.1:54984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7168 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7169 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7207 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7208 T833 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:17765_pun%2Fz
[junit4:junit4]   2> 7209 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:17765_pun%2Fz Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:17765_pun%2Fz
[junit4:junit4]   2> 7214 T833 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:17765_pun%2Fz
[junit4:junit4]   2> 7217 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7218 T874 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7218 T861 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7218 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7218 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7219 T855 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7222 T833 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7229 T890 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360146939225/collection1
[junit4:junit4]   2> 7229 T890 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7230 T890 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7230 T890 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7232 T890 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360146939225/collection1/'
[junit4:junit4]   2> 7233 T890 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360146939225/collection1/lib/README' to classloader
[junit4:junit4]   2> 7233 T890 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360146939225/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7285 T890 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7369 T890 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7470 T890 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7476 T890 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8087 T890 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8099 T890 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8103 T890 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8119 T890 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8124 T890 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8128 T890 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8130 T890 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8131 T890 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360146939225/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/
[junit4:junit4]   2> 8131 T890 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ee76fcc
[junit4:junit4]   2> 8132 T890 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 8133 T890 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2 forceNew:false
[junit4:junit4]   2> 8133 T890 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2
[junit4:junit4]   2> 8133 T890 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/index/
[junit4:junit4]   2> 8134 T890 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8135 T890 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/index forceNew:false
[junit4:junit4]   2> 8149 T890 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c32a60e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8150 T890 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8151 T890 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/index
[junit4:junit4]   2> 8151 T890 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8153 T890 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8153 T890 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8154 T890 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8154 T890 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8155 T890 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8155 T890 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8155 T890 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8156 T890 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8157 T890 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8157 T890 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8158 T890 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8158 T890 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8159 T890 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8159 T890 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8159 T890 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8160 T890 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8160 T890 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8161 T890 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8162 T890 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8162 T890 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8163 T890 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8163 T890 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8164 T890 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8164 T890 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8165 T890 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8165 T890 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8178 T890 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8183 T890 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8184 T890 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8186 T890 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2
[junit4:junit4]   2> 8188 T890 oass.SolrIndexSearcher.<init> Opening Searcher@7a19134a main
[junit4:junit4]   2> 8189 T890 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8189 T890 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8190 T890 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8197 T890 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8255 T891 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a19134a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8261 T890 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8261 T890 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8266 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8267 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50938_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50938/pun/z"}
[junit4:junit4]   2> 8271 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:17765_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:17765/pun/z"}
[junit4:junit4]   2> 8272 T856 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 8272 T856 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 8283 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8283 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8283 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8283 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9264 T890 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9264 T890 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:17765/pun/z collection:collection1 shard:shard2
[junit4:junit4]   2> 9265 T890 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9284 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40008 type:delete cxid:0x42 zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9285 T890 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9286 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40008 type:create cxid:0x43 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9289 T890 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9289 T890 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9289 T890 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:17765/pun/z/collection1/
[junit4:junit4]   2> 9289 T890 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9290 T890 oasc.SyncStrategy.syncToMe http://127.0.0.1:17765/pun/z/collection1/ has no replicas
[junit4:junit4]   2> 9290 T890 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:17765/pun/z/collection1/
[junit4:junit4]   2> 9290 T890 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9294 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c40008 type:create cxid:0x4c zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9796 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9817 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9817 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9817 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9817 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9857 T890 oasc.ZkController.register We are http://127.0.0.1:17765/pun/z/collection1/ and leader is http://127.0.0.1:17765/pun/z/collection1/
[junit4:junit4]   2> 9857 T890 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:17765/pun/z
[junit4:junit4]   2> 9857 T890 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9857 T890 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9858 T890 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9859 T890 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9860 T833 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9861 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9861 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10042 T833 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10046 T833 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44899
[junit4:junit4]   2> 10046 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10047 T833 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10047 T833 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360146942286
[junit4:junit4]   2> 10048 T833 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360146942286/solr.xml
[junit4:junit4]   2> 10048 T833 oasc.CoreContainer.<init> New CoreContainer 1870840746
[junit4:junit4]   2> 10049 T833 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360146942286/'
[junit4:junit4]   2> 10050 T833 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360146942286/'
[junit4:junit4]   2> 10104 T833 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10105 T833 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10105 T833 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10105 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10106 T833 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10106 T833 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10107 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10107 T833 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10108 T833 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10108 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10131 T833 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10151 T833 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54984/solr
[junit4:junit4]   2> 10151 T833 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10152 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3790ea6a
[junit4:junit4]   2> 10153 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10153 T902 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10154 T902 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 10154 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:14629
[junit4:junit4]   2> 10155 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:14629
[junit4:junit4]   2> 10158 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c40009 with negotiated timeout 20000 for client /140.211.11.196:14629
[junit4:junit4]   2> 10158 T902 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c40009, negotiated timeout = 20000
[junit4:junit4]   2> 10158 T903 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3790ea6a name:ZooKeeperConnection Watcher:127.0.0.1:54984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10158 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10159 T838 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caf1322c40009
[junit4:junit4]   2> 10173 T903 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10173 T835 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:14629 which had sessionid 0x13caf1322c40009
[junit4:junit4]   2> 10173 T833 oaz.ZooKeeper.close Session: 0x13caf1322c40009 closed
[junit4:junit4]   2> 10174 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10187 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@151b68a1
[junit4:junit4]   2> 10188 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10188 T904 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10189 T904 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 10189 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:14837
[junit4:junit4]   2> 10189 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:14837
[junit4:junit4]   2> 10191 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c4000a with negotiated timeout 20000 for client /140.211.11.196:14837
[junit4:junit4]   2> 10191 T904 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c4000a, negotiated timeout = 20000
[junit4:junit4]   2> 10191 T905 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@151b68a1 name:ZooKeeperConnection Watcher:127.0.0.1:54984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10191 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10192 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c4000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10194 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c4000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10202 T833 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44899_pun%2Fz
[junit4:junit4]   2> 10203 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c4000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:44899_pun%2Fz Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:44899_pun%2Fz
[junit4:junit4]   2> 10204 T833 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44899_pun%2Fz
[junit4:junit4]   2> 10206 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10207 T874 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10207 T861 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10207 T889 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10207 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10207 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10207 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10208 T855 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10211 T833 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10218 T906 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360146942286/collection1
[junit4:junit4]   2> 10219 T906 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10219 T906 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10220 T906 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10221 T906 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360146942286/collection1/'
[junit4:junit4]   2> 10222 T906 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360146942286/collection1/lib/README' to classloader
[junit4:junit4]   2> 10223 T906 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360146942286/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10275 T906 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10347 T906 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10448 T906 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10454 T906 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11072 T906 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11085 T906 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11089 T906 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11117 T906 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11123 T906 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11127 T906 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11129 T906 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11130 T906 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360146942286/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/
[junit4:junit4]   2> 11130 T906 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ee76fcc
[junit4:junit4]   2> 11131 T906 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 11132 T906 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3 forceNew:false
[junit4:junit4]   2> 11133 T906 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 11133 T906 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index/
[junit4:junit4]   2> 11133 T906 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11134 T906 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index forceNew:false
[junit4:junit4]   2> 11141 T906 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@ed3e462; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11141 T906 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11142 T906 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index
[junit4:junit4]   2> 11142 T906 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11144 T906 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11145 T906 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11145 T906 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11145 T906 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11146 T906 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11146 T906 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11147 T906 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11147 T906 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11148 T906 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11149 T906 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11149 T906 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11150 T906 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11150 T906 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11150 T906 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11151 T906 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11151 T906 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11152 T906 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11152 T906 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11153 T906 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11153 T906 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11154 T906 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11155 T906 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11155 T906 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11156 T906 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11156 T906 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11157 T906 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11169 T906 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11174 T906 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11175 T906 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11178 T906 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 11180 T906 oass.SolrIndexSearcher.<init> Opening Searcher@6f6cedb1 main
[junit4:junit4]   2> 11181 T906 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11181 T906 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11181 T906 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11188 T906 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11243 T907 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f6cedb1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11249 T906 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11249 T906 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11332 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11333 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:17765_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:17765/pun/z"}
[junit4:junit4]   2> 11338 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44899_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44899/pun/z"}
[junit4:junit4]   2> 11338 T856 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 11338 T856 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11350 T905 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11350 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11350 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11350 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11350 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12252 T906 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12252 T906 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44899/pun/z collection:collection1 shard:shard1
[junit4:junit4]   2> 12262 T906 oasc.ZkController.register We are http://127.0.0.1:44899/pun/z/collection1/ and leader is http://127.0.0.1:50938/pun/z/collection1/
[junit4:junit4]   2> 12262 T906 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44899/pun/z
[junit4:junit4]   2> 12262 T906 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 12262 T906 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C44 name=collection1 org.apache.solr.core.SolrCore@20f2375d url=http://127.0.0.1:44899/pun/z/collection1 node=127.0.0.1:44899_pun%2Fz C44_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:44899_pun%2Fz, base_url=http://127.0.0.1:44899/pun/z}
[junit4:junit4]   2> 12269 T908 C44 P44899 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 12270 T906 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12271 T908 C44 P44899 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 12271 T908 C44 P44899 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 12271 T908 C44 P44899 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12272 T833 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12272 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12273 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12273 T908 C44 P44899 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12468 T833 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12471 T833 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:61009
[junit4:junit4]   2> 12472 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12472 T833 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12473 T833 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360146944707
[junit4:junit4]   2> 12473 T833 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360146944707/solr.xml
[junit4:junit4]   2> 12474 T833 oasc.CoreContainer.<init> New CoreContainer 609903698
[junit4:junit4]   2> 12475 T833 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360146944707/'
[junit4:junit4]   2> 12475 T833 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360146944707/'
[junit4:junit4]   2> 12529 T833 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 12530 T833 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12530 T833 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12531 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12531 T833 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12532 T833 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12532 T833 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12533 T833 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12533 T833 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12534 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12557 T833 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12576 T833 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54984/solr
[junit4:junit4]   2> 12577 T833 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12577 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1e791fe2
[junit4:junit4]   2> 12578 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12579 T918 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12579 T918 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 12580 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:37658
[junit4:junit4]   2> 12580 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:37658
[junit4:junit4]   2> 12612 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c4000b with negotiated timeout 20000 for client /140.211.11.196:37658
[junit4:junit4]   2> 12612 T918 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c4000b, negotiated timeout = 20000
[junit4:junit4]   2> 12613 T919 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e791fe2 name:ZooKeeperConnection Watcher:127.0.0.1:54984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12613 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12614 T838 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caf1322c4000b
[junit4:junit4]   2> 12626 T919 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12626 T835 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:37658 which had sessionid 0x13caf1322c4000b
[junit4:junit4]   2> 12626 T833 oaz.ZooKeeper.close Session: 0x13caf1322c4000b closed
[junit4:junit4]   2> 12627 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12641 T833 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54984/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3187829b
[junit4:junit4]   2> 12642 T833 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12642 T920 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54984. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12642 T920 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54984, initiating session
[junit4:junit4]   2> 12643 T835 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:51734
[junit4:junit4]   2> 12643 T835 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:51734
[junit4:junit4]   2> 12644 T837 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caf1322c4000c with negotiated timeout 20000 for client /140.211.11.196:51734
[junit4:junit4]   2> 12644 T920 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54984, sessionid = 0x13caf1322c4000c, negotiated timeout = 20000
[junit4:junit4]   2> 12645 T921 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3187829b name:ZooKeeperConnection Watcher:127.0.0.1:54984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12645 T833 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12646 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c4000c type:create cxid:0x1 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12647 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c4000c type:create cxid:0x2 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12654 T833 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61009_pun%2Fz
[junit4:junit4]   2> 12655 T838 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caf1322c4000c type:delete cxid:0x4 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:61009_pun%2Fz Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:61009_pun%2Fz
[junit4:junit4]   2> 12656 T833 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:61009_pun%2Fz
[junit4:junit4]   2> 12659 T905 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12659 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12660 T874 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12660 T861 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12660 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12660 T889 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12660 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12660 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12661 T855 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12661 T905 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12664 T833 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12671 T922 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360146944707/collection1
[junit4:junit4]   2> 12671 T922 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12672 T922 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12672 T922 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12674 T922 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360146944707/collection1/'
[junit4:junit4]   2> 12675 T922 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360146944707/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12676 T922 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360146944707/collection1/lib/README' to classloader
[junit4:junit4]   2> 12728 T922 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12800 T922 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12865 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12866 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44899_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44899/pun/z"}
[junit4:junit4]   2> 12880 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12880 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12880 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12880 T905 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12880 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12901 T922 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12901 T921 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12908 T922 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13514 T922 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13525 T922 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13529 T922 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13544 T922 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13549 T922 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13553 T922 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13554 T922 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13556 T922 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360146944707/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/
[junit4:junit4]   2> 13556 T922 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ee76fcc
[junit4:junit4]   2> 13556 T922 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 13557 T922 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4 forceNew:false
[junit4:junit4]   2> 13558 T922 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 13558 T922 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index/
[junit4:junit4]   2> 13559 T922 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13560 T922 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index forceNew:false
[junit4:junit4]   2> 13591 T922 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@39f1581; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13591 T922 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13592 T922 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index
[junit4:junit4]   2> 13592 T922 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13594 T922 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13595 T922 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13595 T922 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13595 T922 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13596 T922 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13596 T922 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13596 T922 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13597 T922 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13598 T922 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13598 T922 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13599 T922 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13599 T922 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13600 T922 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13600 T922 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13600 T922 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13601 T922 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13601 T922 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13602 T922 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13602 T922 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13603 T922 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13603 T922 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13604 T922 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13604 T922 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13605 T922 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13605 T922 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13606 T922 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13618 T922 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13622 T922 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13623 T922 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13626 T922 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 13627 T922 oass.SolrIndexSearcher.<init> Opening Searcher@44ee4044 main
[junit4:junit4]   2> 13628 T922 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13629 T922 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13629 T922 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13636 T922 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13689 T923 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44ee4044 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13694 T922 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13694 T922 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14385 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14386 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:61009_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:61009/pun/z"}
[junit4:junit4]   2> 14386 T856 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14386 T856 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 14434 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14434 T921 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14434 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14434 T905 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14434 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14434 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14697 T922 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 14697 T922 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:61009/pun/z collection:collection1 shard:shard2
[junit4:junit4]   2> 14700 T922 oasc.ZkController.register We are http://127.0.0.1:61009/pun/z/collection1/ and leader is http://127.0.0.1:17765/pun/z/collection1/
[junit4:junit4]   2> 14700 T922 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:61009/pun/z
[junit4:junit4]   2> 14701 T922 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 14701 T922 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C45 name=collection1 org.apache.solr.core.SolrCore@15988fe8 url=http://127.0.0.1:61009/pun/z/collection1 node=127.0.0.1:61009_pun%2Fz C45_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:61009_pun%2Fz, base_url=http://127.0.0.1:61009/pun/z}
[junit4:junit4]   2> 14701 T924 C45 P61009 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 14702 T922 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14702 T924 C45 P61009 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 14703 T924 C45 P61009 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 14703 T924 C45 P61009 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14703 T833 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14704 T833 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14705 T833 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14705 T924 C45 P61009 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14723 T833 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14726 T833 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 14727 T833 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C44_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:44899_pun%2Fz, base_url=http://127.0.0.1:44899/pun/z}
[junit4:junit4]   2> 15323 T908 C44 P44899 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50938/pun/z/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15333 T908 C44 P44899 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 15351 T908 C44 P44899 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44899/pun/z START replicas=[http://127.0.0.1:50938/pun/z/collection1/] nUpdates=100
[junit4:junit4]   2> 15352 T908 C44 P44899 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15353 T908 C44 P44899 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15353 T908 C44 P44899 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15353 T908 C44 P44899 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15354 T908 C44 P44899 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15354 T908 C44 P44899 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50938/pun/z/collection1/. core=collection1
[junit4:junit4]   2> 15354 T908 C44 P44899 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C46 name=collection1 org.apache.solr.core.SolrCore@4663e5ec url=http://127.0.0.1:50938/pun/z/collection1 node=127.0.0.1:50938_pun%2Fz C46_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50938_pun%2Fz, base_url=http://127.0.0.1:50938/pun/z, leader=true}
[junit4:junit4]   2> 15364 T865 C46 P50938 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15384 T866 C46 P50938 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15385 T866 C46 P50938 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1
[junit4:junit4]   2> 15390 T866 C46 P50938 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e187eda; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15391 T866 C46 P50938 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15395 T866 C46 P50938 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e187eda; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e187eda; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15395 T866 C46 P50938 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15396 T866 C46 P50938 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1
[junit4:junit4]   2> 15397 T866 C46 P50938 oass.SolrIndexSearcher.<init> Opening Searcher@37584f32 realtime
[junit4:junit4]   2> 15397 T866 C46 P50938 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15398 T866 C46 P50938 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 31
[junit4:junit4]   2> 15399 T908 C44 P44899 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15399 T908 C44 P44899 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 15426 T908 C44 P44899 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15428 T867 C46 P50938 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15429 T867 C46 P50938 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15430 T908 C44 P44899 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15430 T908 C44 P44899 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15431 T908 C44 P44899 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15434 T867 C46 P50938 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1
[junit4:junit4]   2> 15434 T867 C46 P50938 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1/index
[junit4:junit4]   2> 15435 T867 C46 P50938 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15436 T908 C44 P44899 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15437 T908 C44 P44899 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index.20130205233547845 forceNew:false
[junit4:junit4]   2> 15438 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 15438 T908 C44 P44899 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index.20130205233547845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e6c9836; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 15452 T867 C46 P50938 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=5 
[junit4:junit4]   2> 15467 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index
[junit4:junit4]   2> 15467 T908 C44 P44899 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15468 T908 C44 P44899 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130205233547845
[junit4:junit4]   2> 15476 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 15477 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 15477 T908 C44 P44899 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index.20130205233547845
[junit4:junit4]   2> 15481 T908 C44 P44899 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index.20130205233547845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e6c9836; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15481 T908 C44 P44899 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15483 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 15487 T908 C44 P44899 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15488 T908 C44 P44899 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15488 T908 C44 P44899 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 15489 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index.20130205233547845
[junit4:junit4]   2> 15489 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 15493 T908 C44 P44899 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index.20130205233547845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e6c9836; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15493 T908 C44 P44899 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15493 T908 C44 P44899 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15494 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 15494 T908 C44 P44899 oass.SolrIndexSearcher.<init> Opening Searcher@25cd4e12 main
[junit4:junit4]   2> 15496 T907 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25cd4e12 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15497 T907 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index
[junit4:junit4]   2> 15497 T908 C44 P44899 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15509 T908 C44 P44899 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index.20130205233547845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e6c9836; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index.20130205233547845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e6c9836; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 15509 T908 C44 P44899 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 15510 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 15511 T908 C44 P44899 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15511 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index.20130205233547845
[junit4:junit4]   2> 15511 T908 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3/index
[junit4:junit4]   2> 15511 T908 C44 P44899 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15512 T908 C44 P44899 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15512 T908 C44 P44899 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15512 T908 C44 P44899 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15514 T908 C44 P44899 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 15729 T833 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15940 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15941 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:61009_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:61009/pun/z"}
[junit4:junit4]   2> 15945 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44899_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44899/pun/z"}
[junit4:junit4]   2> 15957 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15958 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15957 T921 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15957 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15957 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15957 T905 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16731 T833 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17733 T833 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C45_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:61009_pun%2Fz, base_url=http://127.0.0.1:61009/pun/z}
[junit4:junit4]   2> 18728 T924 C45 P61009 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:17765/pun/z/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18728 T924 C45 P61009 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:61009/pun/z START replicas=[http://127.0.0.1:17765/pun/z/collection1/] nUpdates=100
[junit4:junit4]   2> 18729 T924 C45 P61009 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18729 T924 C45 P61009 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18729 T924 C45 P61009 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18729 T924 C45 P61009 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18729 T924 C45 P61009 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18730 T924 C45 P61009 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:17765/pun/z/collection1/. core=collection1
[junit4:junit4]   2> 18731 T924 C45 P61009 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18735 T833 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C47 name=collection1 org.apache.solr.core.SolrCore@370d9885 url=http://127.0.0.1:17765/pun/z/collection1 node=127.0.0.1:17765_pun%2Fz C47_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:17765_pun%2Fz, base_url=http://127.0.0.1:17765/pun/z, leader=true}
[junit4:junit4]   2> 18741 T880 C47 P17765 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18749 T881 C47 P17765 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18750 T881 C47 P17765 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2
[junit4:junit4]   2> 18754 T881 C47 P17765 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c32a60e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18754 T881 C47 P17765 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18758 T881 C47 P17765 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c32a60e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c32a60e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18759 T881 C47 P17765 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18760 T881 C47 P17765 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2
[junit4:junit4]   2> 18760 T881 C47 P17765 oass.SolrIndexSearcher.<init> Opening Searcher@1e7aac6e realtime
[junit4:junit4]   2> 18761 T881 C47 P17765 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18761 T881 C47 P17765 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4]   2> 18762 T924 C45 P61009 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18763 T924 C45 P61009 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18765 T882 C47 P17765 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18766 T882 C47 P17765 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18767 T924 C45 P61009 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18767 T924 C45 P61009 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18767 T924 C45 P61009 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18770 T882 C47 P17765 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2
[junit4:junit4]   2> 18770 T882 C47 P17765 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2/index
[junit4:junit4]   2> 18771 T882 C47 P17765 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18772 T924 C45 P61009 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18773 T924 C45 P61009 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index.20130205233551181 forceNew:false
[junit4:junit4]   2> 18773 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 18774 T924 C45 P61009 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index.20130205233551181 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7007d61a; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 18777 T882 C47 P17765 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 18779 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index
[junit4:junit4]   2> 18779 T924 C45 P61009 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18780 T924 C45 P61009 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130205233551181
[junit4:junit4]   2> 18781 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 18782 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 18782 T924 C45 P61009 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index.20130205233551181
[junit4:junit4]   2> 18785 T924 C45 P61009 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index.20130205233551181 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7007d61a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18786 T924 C45 P61009 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18787 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 18788 T924 C45 P61009 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18788 T924 C45 P61009 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18788 T924 C45 P61009 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18789 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index.20130205233551181
[junit4:junit4]   2> 18790 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 18793 T924 C45 P61009 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index.20130205233551181 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7007d61a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18793 T924 C45 P61009 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18793 T924 C45 P61009 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18794 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 18794 T924 C45 P61009 oass.SolrIndexSearcher.<init> Opening Searcher@2f264d4d main
[junit4:junit4]   2> 18796 T923 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f264d4d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18797 T923 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index
[junit4:junit4]   2> 18797 T924 C45 P61009 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18800 T924 C45 P61009 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index.20130205233551181 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7007d61a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index.20130205233551181 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7007d61a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18800 T924 C45 P61009 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18802 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 18802 T924 C45 P61009 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18802 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index.20130205233551181
[junit4:junit4]   2> 18802 T924 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4/index
[junit4:junit4]   2> 18802 T924 C45 P61009 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18803 T924 C45 P61009 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18803 T924 C45 P61009 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18803 T924 C45 P61009 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18810 T924 C45 P61009 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 18973 T856 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18974 T856 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:61009_pun%2Fz",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:61009/pun/z"}
[junit4:junit4]   2> 18989 T905 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18989 T855 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18989 T921 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18989 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18989 T874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18989 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19737 T833 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19738 T833 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C48 name=collection1 org.apache.solr.core.SolrCore@7484ffe2 url=http://127.0.0.1:62803/pun/z/collection1 node=127.0.0.1:62803_pun%2Fz C48_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:62803_pun%2Fz, base_url=http://127.0.0.1:62803/pun/z, leader=true}
[junit4:junit4]   2> 19767 T845 C48 P62803 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data
[junit4:junit4]   2> 19771 T845 C48 P62803 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6938af32; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19772 T845 C48 P62803 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19777 T845 C48 P62803 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/control/data
[junit4:junit4]   2> 19778 T845 C48 P62803 oass.SolrIndexSearcher.<init> Opening Searcher@69b09ef0 realtime
[junit4:junit4]   2> 19778 T845 C48 P62803 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1426217450512318464)} 0 27
[junit4:junit4]   2> 19791 T883 C47 P17765 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty2
[junit4:junit4]   2> 19812 T868 C46 P50938 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty1
[junit4:junit4]   2>  C45_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:61009_pun%2Fz, base_url=http://127.0.0.1:61009/pun/z}
[junit4:junit4]   2> 19820 T911 C45 P61009 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty4
[junit4:junit4]   2> 19821 T911 C45 P61009 /update {update.distrib=FROMLEADER&_version_=-1426217450547970048&update.from=http://127.0.0.1:17765/pun/z/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1426217450547970048)} 0 3
[junit4:junit4]   2>  C44_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44899_pun%2Fz, base_url=http://127.0.0.1:44899/pun/z}
[junit4:junit4]   2> 19826 T895 C44 P44899 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360146932413/jetty3
[junit4:junit4]   2> 19827 T895 C44 P44899 /update {update.distrib=FROMLEADER&_version_=-1426217450571038720&update.from=http://127.0.0.1:50938/pun/z/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1426217450571038720)} 0 3
[junit4:junit4]   2> 19828 T868 C46 P50938 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1426217450571038720)} 0 18
[junit4:junit4]   2> 19828 T883 C47 P17765 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1426217450547970048)} 0 47
[junit4:junit4]   2> 19837 T845 C48 P62803 /update {wt=javabin&version=2} {add=[1 (1426217450594107392)]} 0 5
[junit4:junit4]   2> 19867 T895 C44 P44899 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1426217450620321792)]} 0 4
[junit4:junit4]   2> 19868 T868 C46 P50938 /update {distrib.from=http://127.0.0.1:17765/pun/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1426217450620321792)]} 0 11
[junit4:junit4]   2> 19869 T883 C47 P17765 /update {wt=javabin&version=2} {add=[1]} 0 29
[junit4:junit4]   2> 19874 T845 C48 P62803 /update {wt=javabin&version=2} {add=[2 (1426217450636050432)]} 0 2
[junit4:junit4]   2> 19889 T911 C45 P61009 /update {distrib.from=http://127.0.0.1:17765/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1426217450642341888)]} 0 4
[junit4:junit4]   2> 19890 T884 C47 P17765 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1426217450642341888)]} 0 11
[junit4:junit4]   2> 19890 T869 C46 P50938 /update {wt=javabin&version=2} {add=[2]} 0 13
[junit4:junit4]   2> 19896 T845 C48 P62803 /update {wt=javabin&version=2} {add=[3 (1426217450659119104)]} 0 2
[junit4:junit4]   2> 19908 T911 C45 P61009 /update {distrib.from=http://127.0.0.1:17765/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1426217450667507712)]} 0 2
[junit4:junit4]   2> 19909 T884 C47 P17765 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1426217450667507712)]} 0 7
[junit4:junit4]   2> 19910 T869 C46 P50938 /update {wt=javabin&version=2} {add=[3]} 0 11
[junit4:junit4]   2> 19915 T845 C48 P62803 /update {wt=javabin&version=2} {add=[4 (1426217450679042048)]} 0 2
[junit4:junit4]   2> 19925 T895 C44 P44899 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1426217450684284928)]} 0 2
[junit4:junit4]   2> 19926 T869 C46 P50938 /update {wt=javabin&version=2} {add=[4 (1426217450684284928)]} 0 8
[junit4:junit4]   2> 19931 T845 C48 P62803 /update {wt=javabin&version=2} {add=[5 (1426217450695819264)]} 0 2
[junit4:junit4]   2> 19943 T911 C45 P61009 /update {distrib.from=http://127.0.0.1:17765/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1426217450704207872)]} 0 2
[junit4:junit4]   2> 19944 T884 C47 P17765 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1426217450704207872)]} 0 7
[junit4:junit4]   2> 19945 T869 C46 P50938 /update {wt=javabin&version=2} {add=[5]} 0 11
[junit4:junit4]   2> 19950 T845 C48 P62803 /update {wt=javabin&version=2} {add=[6 (1426217450715742208)]} 0 2
[junit4:junit4]   2> 19962 T911 C45 P61009 /update {distrib.from=http://127.0.0.1:17765/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1426217450724130816)]} 0 2
[junit4:junit4]   2> 19963 T884 C47 P17765 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1426217450724130816)]} 0 7
[junit4:junit4]   2> 19964 T869 C46 P50938 /update {wt=javabin&version=2} {add=[6]} 0 11
[junit4:junit4]   2> 19969 T845 C48 P62803 /update {wt=javabin&version=2} {add=[7 (1426217450735665152)]} 0 2
[junit4:junit4]   2> 19979 T911 C45 P61009 /update {distrib.from=http://127.0.0.1:17765/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1426217450740908032)]} 0 2
[junit4:junit4]   2> 19980 T883 C47 P17765 /update {wt=javabin&version=2} {add=[7 (1426217450740908032)]} 0 8
[junit4:junit4]   2> 19985 T845 C48 P62803 /update {wt=javabin&version=2} {add=[8 (1426217450752442368)]} 0 2
[junit4:junit4]   2> 19998 T895 C44 P44899 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1426217450760830976)]} 0 2
[junit4:junit4]   2> 19999 T868 C46 P50938 /update {distrib.from=http://127.0.0.1:17765/pun/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1426217450760830976)]} 0 8
[junit4:junit4]   2> 20000 T883 C47 P17765 /update {wt=javabin&version=2} {add=[8]} 0 12
[junit4:junit4]   2> 20005 T845 C48 P62803 /update {wt=javabin&version=2} {add=[9 (1426217450773413888)]} 0 2
[junit4:junit4]   2> 20018 T911 C45 P61009 /update {distrib.from=http://127.0.0.1:17765/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1426217450781802496)]} 0 2
[junit4:junit4]   2> 20019 T884 C47 P17765 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1426217450781802496)]} 0 8
[junit4:junit4]   2> 20020 T869 C46 P50938 /update {wt=javabin&version=2} {add=[9]} 0 12
[junit4:junit4]   2> 20040 T845 C48 P62803 /update {wt=javabin&version=2} {add=[10 (1426217450794385408)]} 0 17
[junit4:junit4]   2> 20049 T895 C44 P44899 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1426217450815356928)]} 0 2
[junit4:junit4]   2> 20050 T869 C46 P50938 /update {wt=javabin&version=2} {add=[10 (1426217450815356928)]} 0 7
[junit4:junit4]   2> 20057 T845 C48 P62803 /update {wt=javabin&version=2} {add=[11 (1426217450825842688)]} 0 4
[junit4:junit4]   2> 20067 T895 C44 P44899 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1426217450833182720)]} 0 2
[junit4:junit4]   2> 20068 T869 C46 P50938 /update {wt=javabin&version=2} {add=[11 (1426217450833182720)]} 0 8
[junit4:junit4]   2> 20073 T845 C48 P62803 /update {wt=javabin&version=2} {add=[12 (1426217450844717056)]} 0 2
[junit4:junit4]   2> 20086 T895 C44 P44899 /update {distrib.from=http://127.0.0.1:50938/pun/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1426217450853105664)]} 0 2
[junit4:junit4]   2> 20087 T868 C46 P50938 /update {distrib.from=http://127.0.0.1:1

[...truncated too long message...]

st]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  26) Thread[id=1300, name=RecoveryThread, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]   2> 	        at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
[junit4:junit4]   2> 	        at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
[junit4:junit4]   2> 	        at org.apache.http.impl.io.SocketInputBuffer.isDataAvailable(SocketInputBuffer.java:122)
[junit4:junit4]   2> 	        at org.apache.http.impl.AbstractHttpClientConnection.isStale(AbstractHttpClientConnection.java:311)
[junit4:junit4]   2> 	        at org.apache.http.impl.conn.AbstractClientConnAdapter.isStale(AbstractClientConnAdapter.java:170)
[junit4:junit4]   2> 	        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:412)
[junit4:junit4]   2> 	        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
[junit4:junit4]   2> 	        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
[junit4:junit4]   2> 	        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
[junit4:junit4]   2> 	        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
[junit4:junit4]   2> 	        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
[junit4:junit4]   2> 	        at org.apache.solr.handler.SnapPuller.fetchFileList(SnapPuller.java:271)
[junit4:junit4]   2> 	        at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:365)
[junit4:junit4]   2> 	        at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:275)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:153)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:223)
[junit4:junit4]   2> 	  27) Thread[id=981, name=qtp407163793-981, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.ByteArrayBuffer.readFrom(ByteArrayBuffer.java:375)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.bio.StreamEndPoint.fill(StreamEndPoint.java:141)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.fill(SocketConnector.java:227)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.fill(HttpParser.java:1035)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:280)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), foo_b=PostingsFormat(name=Memory doPackFST= true), cat=PostingsFormat(name=Direct), foo_d=PostingsFormat(name=MockSep), foo_f=PostingsFormat(name=Memory doPackFST= true), n_tl1=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), n_d1=PostingsFormat(name=Direct), rnd_b=PostingsFormat(name=MockSep), intDefault=PostingsFormat(name=Direct), n_td1=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), timestamp=PostingsFormat(name=Direct), id=PostingsFormat(name=MockSep), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), oddField_s=PostingsFormat(name=Memory doPackFST= true), sequence_i=PostingsFormat(name=Memory doPackFST= true), name=PostingsFormat(name=MockSep), foo_i=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), regex_dup_B_s=PostingsFormat(name=Direct), multiDefault=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), n_tf1=PostingsFormat(name=Direct), n_dt1=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), genre_s=PostingsFormat(name=Memory doPackFST= true), author_t=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), n_ti1=PostingsFormat(name=MockSep), range_facet_l=PostingsFormat(name=Direct), text=PostingsFormat(name=Memory doPackFST= true), _version_=PostingsFormat(name=Direct), val_i=PostingsFormat(name=MockSep), SubjectTerms_mfacet=PostingsFormat(name=Direct), series_t=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), a_t=PostingsFormat(name=Direct), n_tdt1=PostingsFormat(name=Direct), regex_dup_A_s=PostingsFormat(name=Memory doPackFST= true), price=PostingsFormat(name=Memory doPackFST= true), other_tl1=PostingsFormat(name=Direct), n_l1=PostingsFormat(name=Direct), a_si=Pulsing41(freqCutoff=9 minBlockSize=50 maxBlockSize=157), inStock=PostingsFormat(name=Memory doPackFST= true)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=et, timezone=Pacific/Niue
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=28,free=153302664,total=488636416
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SolrCoreCheckLockOnStartupTest, FastVectorHighlighterTest, TestJmxMonitoredMap, TestSuggestSpellingConverter, CircularListTest, WordBreakSolrSpellCheckerTest, TermsComponentTest, QueryElevationComponentTest, SpatialFilterTest, SpellPossibilityIteratorTest, TestPseudoReturnFields, ZkCLITest, JSONWriterTest, TestPHPSerializedResponseWriter, TestConfig, TestSearchPerf, TestIBSimilarityFactory, AutoCommitTest, FileBasedSpellCheckerTest, TestRandomFaceting, TestBadConfig, TestXIncludeConfig, TestPartialUpdateDeduplication, CoreContainerCoreInitFailuresTest, LeaderElectionIntegrationTest, ZkControllerTest, NotRequiredUniqueKeyTest, DefaultValueUpdateProcessorTest, TestCharFilters, CSVRequestHandlerTest, IndexBasedSpellCheckerTest, TestFaceting, NumericFieldsTest, DateFieldTest, TestDefaultSimilarityFactory, HighlighterConfigTest, TestFuzzyAnalyzedSuggestions, TestIndexSearcher, DirectSolrConnectionTest, ClusterStateUpdateTest, MinimalSchemaTest, TestLazyCores, DistributedSpellCheckComponentTest, TestLMJelinekMercerSimilarityFactory, SystemInfoHandlerTest, TestDocumentBuilder, TestWordDelimiterFilterFactory, FieldAnalysisRequestHandlerTest, SampleTest, TestGroupingSearch, TestElisionMultitermQuery, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=F7C2F0E9117976D6 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=et -Dtests.timezone=Pacific/Niue -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([F7C2F0E9117976D6]:0)
[junit4:junit4] Completed on J0 in 7223.67s, 1 test, 2 errors <<< FAILURES!

[...truncated 35 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:432: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 32 ignored (4 assumptions)

Total time: 156 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3714 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3714/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
shard1 should have just been set up to be inconsistent - but it's still consistent

Stack Trace:
java.lang.AssertionError: shard1 should have just been set up to be inconsistent - but it's still consistent
	at __randomizedtesting.SeedInfo.seed([4102611226DCA49D:C0E4EF0A5183C4A1]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:214)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:57102/dk_vlc/o/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57102/dk_vlc/o/collection1
	at __randomizedtesting.SeedInfo.seed([4102611226DCA49D:C0E4EF0A5183C4A1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168)
	at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:146)
	at org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:889)
	at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:334)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
	... 47 more




Build Log:
[...truncated 16249 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T204 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /dk_vlc/o
[junit4:junit4]   2> 7 T204 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1360188203821
[junit4:junit4]   2> 9 T204 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 10 T205 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 13 T205 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 13 T205 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 13 T205 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 14 T205 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 15 T205 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 111 T204 oasc.ZkTestServer.run start zk server on port:57059
[junit4:junit4]   2> 111 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6e37de6a
[junit4:junit4]   2> 112 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 113 T210 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 113 T210 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 114 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57060
[junit4:junit4]   2> 114 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57060
[junit4:junit4]   2> 115 T208 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 117 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340000 with negotiated timeout 10000 for client /140.211.11.196:57060
[junit4:junit4]   2> 117 T210 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340000, negotiated timeout = 10000
[junit4:junit4]   2> 117 T211 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e37de6a name:ZooKeeperConnection Watcher:127.0.0.1:57059 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 118 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 118 T204 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 121 T209 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cb188e3340000
[junit4:junit4]   2> 130 T211 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 130 T204 oaz.ZooKeeper.close Session: 0x13cb188e3340000 closed
[junit4:junit4]   2> 131 T206 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57060 which had sessionid 0x13cb188e3340000
[junit4:junit4]   2> 131 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@357f908
[junit4:junit4]   2> 132 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 133 T212 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 133 T212 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 133 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57061
[junit4:junit4]   2> 134 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57061
[junit4:junit4]   2> 135 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340001 with negotiated timeout 10000 for client /140.211.11.196:57061
[junit4:junit4]   2> 135 T212 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340001, negotiated timeout = 10000
[junit4:junit4]   2> 135 T213 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@357f908 name:ZooKeeperConnection Watcher:127.0.0.1:57059/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 136 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 136 T204 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 154 T204 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 165 T204 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 168 T204 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 180 T204 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 181 T204 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 188 T204 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 189 T204 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 294 T204 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 295 T204 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 309 T204 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 310 T204 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 323 T204 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 323 T204 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 335 T204 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 336 T204 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 346 T204 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 347 T204 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 354 T204 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 355 T204 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 362 T204 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 363 T204 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 370 T209 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cb188e3340001
[junit4:junit4]   2> 372 T213 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 372 T206 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57061 which had sessionid 0x13cb188e3340001
[junit4:junit4]   2> 372 T204 oaz.ZooKeeper.close Session: 0x13cb188e3340001 closed
[junit4:junit4]   2> 1132 T204 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 1350 T204 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57065
[junit4:junit4]   2> 1386 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1387 T204 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1387 T204 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360188204185
[junit4:junit4]   2> 1387 T204 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360188204185/solr.xml
[junit4:junit4]   2> 1388 T204 oasc.CoreContainer.<init> New CoreContainer 409192685
[junit4:junit4]   2> 1388 T204 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360188204185/'
[junit4:junit4]   2> 1389 T204 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360188204185/'
[junit4:junit4]   2> 1426 T204 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 1426 T204 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1427 T204 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 1427 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1427 T204 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1428 T204 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1428 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1428 T204 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1429 T204 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1429 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 1445 T204 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1459 T204 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57059/solr
[junit4:junit4]   2> 1460 T204 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1460 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4d3ccab7
[junit4:junit4]   2> 1461 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1462 T224 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1463 T224 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 1463 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57066
[junit4:junit4]   2> 1464 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57066
[junit4:junit4]   2> 1465 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340002 with negotiated timeout 20000 for client /140.211.11.196:57066
[junit4:junit4]   2> 1465 T224 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340002, negotiated timeout = 20000
[junit4:junit4]   2> 1466 T225 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d3ccab7 name:ZooKeeperConnection Watcher:127.0.0.1:57059 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1466 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1467 T209 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cb188e3340002
[junit4:junit4]   2> 1469 T225 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 1469 T204 oaz.ZooKeeper.close Session: 0x13cb188e3340002 closed
[junit4:junit4]   2> 1469 T206 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57066 which had sessionid 0x13cb188e3340002
[junit4:junit4]   2> 1470 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 1481 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@72f99d1b
[junit4:junit4]   2> 1482 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1483 T226 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1484 T226 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 1484 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57067
[junit4:junit4]   2> 1484 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57067
[junit4:junit4]   2> 1525 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340003 with negotiated timeout 20000 for client /140.211.11.196:57067
[junit4:junit4]   2> 1526 T226 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340003, negotiated timeout = 20000
[junit4:junit4]   2> 1526 T227 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72f99d1b name:ZooKeeperConnection Watcher:127.0.0.1:57059/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1526 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1566 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1570 T204 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1582 T204 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57065_dk_vlc%2Fo
[junit4:junit4]   2> 1583 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57065_dk_vlc%2Fo Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57065_dk_vlc%2Fo
[junit4:junit4]   2> 1584 T204 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57065_dk_vlc%2Fo
[junit4:junit4]   2> 1602 T204 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1667 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 1690 T204 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1693 T204 oasc.Overseer.start Overseer (id=89141294126071811-127.0.0.1:57065_dk_vlc%2Fo-n_0000000000) starting
[junit4:junit4]   2> 1694 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1738 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1740 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1742 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1773 T229 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 1774 T204 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1792 T204 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1794 T228 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1798 T230 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360188204185/collection1
[junit4:junit4]   2> 1799 T230 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 1799 T230 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1800 T230 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 1801 T230 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360188204185/collection1/'
[junit4:junit4]   2> 1803 T230 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360188204185/collection1/lib/README' to classloader
[junit4:junit4]   2> 1803 T230 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360188204185/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1858 T230 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1933 T230 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2034 T230 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2044 T230 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2674 T230 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2690 T230 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2693 T230 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2711 T230 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2716 T230 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2719 T230 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2721 T230 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2722 T230 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360188204185/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/control/data/
[junit4:junit4]   2> 2723 T230 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d8307be
[junit4:junit4]   2> 2723 T230 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 2750 T230 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/control/data forceNew:false
[junit4:junit4]   2> 2751 T230 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/control/data
[junit4:junit4]   2> 2751 T230 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/control/data/index/
[junit4:junit4]   2> 2751 T230 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2752 T230 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/control/data/index forceNew:false
[junit4:junit4]   2> 2763 T230 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c4f00de; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2763 T230 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 2764 T230 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/control/data/index
[junit4:junit4]   2> 2764 T230 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2766 T230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2766 T230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2766 T230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2766 T230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2767 T230 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2767 T230 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2767 T230 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2768 T230 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2768 T230 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2769 T230 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2769 T230 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2769 T230 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2770 T230 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2770 T230 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2770 T230 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2771 T230 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2771 T230 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2771 T230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2772 T230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2772 T230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2773 T230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2773 T230 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2773 T230 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2774 T230 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2774 T230 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2775 T230 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2783 T230 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2786 T230 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2788 T230 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 2790 T230 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/control/data
[junit4:junit4]   2> 2791 T230 oass.SolrIndexSearcher.<init> Opening Searcher@d5e6341 main
[junit4:junit4]   2> 2808 T230 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2808 T230 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2809 T230 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2824 T230 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2870 T231 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d5e6341 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2877 T230 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 2877 T230 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2878 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 3298 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3299 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57065_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57065/dk_vlc/o"}
[junit4:junit4]   2> 3299 T228 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 3305 T228 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 3305 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 3345 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3891 T230 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 3891 T230 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57065/dk_vlc/o collection:control_collection shard:shard1
[junit4:junit4]   2> 3892 T230 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 3902 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 3902 T230 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 3903 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3905 T230 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 3906 T230 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 3906 T230 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57065/dk_vlc/o/collection1/
[junit4:junit4]   2> 3906 T230 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 3907 T230 oasc.SyncStrategy.syncToMe http://127.0.0.1:57065/dk_vlc/o/collection1/ has no replicas
[junit4:junit4]   2> 3907 T230 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57065/dk_vlc/o/collection1/
[junit4:junit4]   2> 3907 T230 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 3912 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4851 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4898 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4934 T230 oasc.ZkController.register We are http://127.0.0.1:57065/dk_vlc/o/collection1/ and leader is http://127.0.0.1:57065/dk_vlc/o/collection1/
[junit4:junit4]   2> 4938 T230 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57065/dk_vlc/o
[junit4:junit4]   2> 4938 T230 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 4938 T230 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 4939 T230 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4941 T230 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4943 T204 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 4943 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5015 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5043 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 5061 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2db68692
[junit4:junit4]   2> 5062 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5062 T232 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5063 T232 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 5063 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57074
[junit4:junit4]   2> 5064 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57074
[junit4:junit4]   2> 5096 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340004 with negotiated timeout 10000 for client /140.211.11.196:57074
[junit4:junit4]   2> 5096 T232 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340004, negotiated timeout = 10000
[junit4:junit4]   2> 5096 T233 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2db68692 name:ZooKeeperConnection Watcher:127.0.0.1:57059/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5096 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5098 T204 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5102 T204 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 5273 T204 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 5277 T204 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57075
[junit4:junit4]   2> 5277 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5278 T204 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5278 T204 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360188208914
[junit4:junit4]   2> 5279 T204 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360188208914/solr.xml
[junit4:junit4]   2> 5279 T204 oasc.CoreContainer.<init> New CoreContainer 1476346951
[junit4:junit4]   2> 5280 T204 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360188208914/'
[junit4:junit4]   2> 5281 T204 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360188208914/'
[junit4:junit4]   2> 5336 T204 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 5337 T204 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5337 T204 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 5338 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5338 T204 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5339 T204 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5339 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5340 T204 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5340 T204 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5341 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 5364 T204 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5383 T204 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57059/solr
[junit4:junit4]   2> 5384 T204 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 5384 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5edafb00
[junit4:junit4]   2> 5386 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5386 T243 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5387 T243 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 5387 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57076
[junit4:junit4]   2> 5388 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57076
[junit4:junit4]   2> 5389 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340005 with negotiated timeout 20000 for client /140.211.11.196:57076
[junit4:junit4]   2> 5390 T243 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340005, negotiated timeout = 20000
[junit4:junit4]   2> 5390 T244 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5edafb00 name:ZooKeeperConnection Watcher:127.0.0.1:57059 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5390 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5392 T209 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cb188e3340005
[junit4:junit4]   2> 5393 T244 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 5393 T206 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57076 which had sessionid 0x13cb188e3340005
[junit4:junit4]   2> 5393 T204 oaz.ZooKeeper.close Session: 0x13cb188e3340005 closed
[junit4:junit4]   2> 5394 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 5407 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7d0bb634
[junit4:junit4]   2> 5408 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5408 T245 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5409 T245 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 5409 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57077
[junit4:junit4]   2> 5410 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57077
[junit4:junit4]   2> 5411 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340006 with negotiated timeout 20000 for client /140.211.11.196:57077
[junit4:junit4]   2> 5411 T245 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340006, negotiated timeout = 20000
[junit4:junit4]   2> 5411 T246 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d0bb634 name:ZooKeeperConnection Watcher:127.0.0.1:57059/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5412 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5413 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5414 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5422 T204 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57075_dk_vlc%2Fo
[junit4:junit4]   2> 5423 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57075_dk_vlc%2Fo Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57075_dk_vlc%2Fo
[junit4:junit4]   2> 5424 T204 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57075_dk_vlc%2Fo
[junit4:junit4]   2> 5427 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 5428 T233 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5429 T227 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5437 T204 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5445 T247 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360188208914/collection1
[junit4:junit4]   2> 5445 T247 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5446 T247 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5446 T247 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5448 T247 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360188208914/collection1/'
[junit4:junit4]   2> 5449 T247 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360188208914/collection1/lib/README' to classloader
[junit4:junit4]   2> 5450 T247 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360188208914/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5505 T247 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 5583 T247 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5684 T247 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5692 T247 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6334 T247 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6348 T247 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6352 T247 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6377 T247 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6382 T247 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6387 T247 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6389 T247 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6391 T247 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360188208914/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/
[junit4:junit4]   2> 6391 T247 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d8307be
[junit4:junit4]   2> 6392 T247 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 6393 T247 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1 forceNew:false
[junit4:junit4]   2> 6393 T247 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1
[junit4:junit4]   2> 6393 T247 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/index/
[junit4:junit4]   2> 6394 T247 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6395 T247 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/index forceNew:false
[junit4:junit4]   2> 6412 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6413 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57065_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57065/dk_vlc/o"}
[junit4:junit4]   2> 6440 T247 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d95aab2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6440 T247 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 6441 T247 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/index
[junit4:junit4]   2> 6442 T247 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 6442 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6442 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6443 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6445 T247 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6446 T247 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 6446 T247 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6446 T247 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 6447 T247 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6447 T247 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6448 T247 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6449 T247 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6449 T247 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6450 T247 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 6451 T247 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 6451 T247 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6452 T247 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 6453 T247 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 6453 T247 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 6454 T247 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6454 T247 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6455 T247 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6456 T247 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6456 T247 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6457 T247 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6458 T247 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6459 T247 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6460 T247 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6460 T247 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 6461 T247 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 6478 T247 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6484 T247 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 6485 T247 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 6488 T247 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1
[junit4:junit4]   2> 6490 T247 oass.SolrIndexSearcher.<init> Opening Searcher@7725310f main
[junit4:junit4]   2> 6491 T247 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6491 T247 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6492 T247 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6550 T247 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6820 T248 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7725310f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6827 T247 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 6827 T247 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7955 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7956 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57075_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57075/dk_vlc/o"}
[junit4:junit4]   2> 7956 T228 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 7956 T228 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 7972 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7972 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7972 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8846 T247 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 8846 T247 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57075/dk_vlc/o collection:collection1 shard:shard1
[junit4:junit4]   2> 8847 T247 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 8855 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 8856 T247 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 8857 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8859 T247 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 8860 T247 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 8860 T247 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57075/dk_vlc/o/collection1/
[junit4:junit4]   2> 8860 T247 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 8860 T247 oasc.SyncStrategy.syncToMe http://127.0.0.1:57075/dk_vlc/o/collection1/ has no replicas
[junit4:junit4]   2> 8861 T247 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57075/dk_vlc/o/collection1/
[junit4:junit4]   2> 8861 T247 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 8866 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9478 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9522 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9522 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9522 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9582 T247 oasc.ZkController.register We are http://127.0.0.1:57075/dk_vlc/o/collection1/ and leader is http://127.0.0.1:57075/dk_vlc/o/collection1/
[junit4:junit4]   2> 9582 T247 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57075/dk_vlc/o
[junit4:junit4]   2> 9582 T247 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9583 T247 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9583 T247 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9586 T247 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9587 T204 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9587 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9588 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9772 T204 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 9775 T204 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57087
[junit4:junit4]   2> 9776 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9777 T204 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9777 T204 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360188213415
[junit4:junit4]   2> 9778 T204 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360188213415/solr.xml
[junit4:junit4]   2> 9778 T204 oasc.CoreContainer.<init> New CoreContainer 914971701
[junit4:junit4]   2> 9779 T204 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360188213415/'
[junit4:junit4]   2> 9780 T204 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360188213415/'
[junit4:junit4]   2> 9836 T204 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 9836 T204 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9837 T204 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 9837 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9838 T204 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9838 T204 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9839 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9839 T204 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9840 T204 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9840 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 9861 T204 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9881 T204 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57059/solr
[junit4:junit4]   2> 9881 T204 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 9882 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@44200eab
[junit4:junit4]   2> 9883 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9884 T258 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9884 T258 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 9885 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57088
[junit4:junit4]   2> 9885 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57088
[junit4:junit4]   2> 9887 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340007 with negotiated timeout 20000 for client /140.211.11.196:57088
[junit4:junit4]   2> 9887 T258 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340007, negotiated timeout = 20000
[junit4:junit4]   2> 9888 T259 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44200eab name:ZooKeeperConnection Watcher:127.0.0.1:57059 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9888 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9889 T209 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cb188e3340007
[junit4:junit4]   2> 9898 T259 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 9899 T206 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57088 which had sessionid 0x13cb188e3340007
[junit4:junit4]   2> 9898 T204 oaz.ZooKeeper.close Session: 0x13cb188e3340007 closed
[junit4:junit4]   2> 9899 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 9911 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6c0fcedd
[junit4:junit4]   2> 9912 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9912 T260 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9913 T260 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 9913 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57089
[junit4:junit4]   2> 9914 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57089
[junit4:junit4]   2> 9915 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340008 with negotiated timeout 20000 for client /140.211.11.196:57089
[junit4:junit4]   2> 9915 T260 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340008, negotiated timeout = 20000
[junit4:junit4]   2> 9915 T261 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c0fcedd name:ZooKeeperConnection Watcher:127.0.0.1:57059/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9916 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9917 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9918 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9925 T204 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57087_dk_vlc%2Fo
[junit4:junit4]   2> 9926 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57087_dk_vlc%2Fo Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57087_dk_vlc%2Fo
[junit4:junit4]   2> 9927 T204 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57087_dk_vlc%2Fo
[junit4:junit4]   2> 9930 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9930 T233 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9930 T246 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9931 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9931 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9932 T227 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9940 T204 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9947 T262 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360188213415/collection1
[junit4:junit4]   2> 9948 T262 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9948 T262 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9948 T262 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9950 T262 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360188213415/collection1/'
[junit4:junit4]   2> 9951 T262 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360188213415/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9952 T262 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360188213415/collection1/lib/README' to classloader
[junit4:junit4]   2> 10006 T262 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10079 T262 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10180 T262 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10188 T262 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10842 T262 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10855 T262 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10859 T262 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10882 T262 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10887 T262 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10892 T262 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10894 T262 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10895 T262 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360188213415/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/
[junit4:junit4]   2> 10896 T262 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d8307be
[junit4:junit4]   2> 10896 T262 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 10897 T262 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2 forceNew:false
[junit4:junit4]   2> 10898 T262 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2
[junit4:junit4]   2> 10898 T262 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/index/
[junit4:junit4]   2> 10899 T262 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10900 T262 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/index forceNew:false
[junit4:junit4]   2> 10906 T262 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7cea9a75; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10907 T262 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 10908 T262 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/index
[junit4:junit4]   2> 10908 T262 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10911 T262 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10911 T262 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10911 T262 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10912 T262 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10912 T262 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10913 T262 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10913 T262 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10914 T262 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10915 T262 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10915 T262 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10916 T262 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10917 T262 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10917 T262 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10917 T262 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10918 T262 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10919 T262 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10919 T262 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10920 T262 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10921 T262 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10921 T262 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10922 T262 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10923 T262 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10923 T262 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10924 T262 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10925 T262 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10926 T262 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10938 T262 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10943 T262 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 10945 T262 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 10947 T262 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2
[junit4:junit4]   2> 10949 T262 oass.SolrIndexSearcher.<init> Opening Searcher@5c696c8b main
[junit4:junit4]   2> 10950 T262 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10951 T262 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10951 T262 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 10958 T262 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11026 T263 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5c696c8b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11034 T262 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11034 T262 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11037 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11038 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57075_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57075/dk_vlc/o"}
[junit4:junit4]   2> 11043 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57087_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57087/dk_vlc/o"}
[junit4:junit4]   2> 11043 T228 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 11043 T228 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 11057 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11057 T261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11057 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11057 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12037 T262 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12037 T262 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57087/dk_vlc/o collection:collection1 shard:shard2
[junit4:junit4]   2> 12038 T262 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 12045 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340008 type:delete cxid:0x42 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 12046 T262 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 12047 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340008 type:create cxid:0x43 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12049 T262 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 12049 T262 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 12050 T262 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57087/dk_vlc/o/collection1/
[junit4:junit4]   2> 12050 T262 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 12050 T262 oasc.SyncStrategy.syncToMe http://127.0.0.1:57087/dk_vlc/o/collection1/ has no replicas
[junit4:junit4]   2> 12051 T262 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57087/dk_vlc/o/collection1/
[junit4:junit4]   2> 12051 T262 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 12055 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e3340008 type:create cxid:0x4c zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12565 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12584 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12584 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12584 T261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12584 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12618 T262 oasc.ZkController.register We are http://127.0.0.1:57087/dk_vlc/o/collection1/ and leader is http://127.0.0.1:57087/dk_vlc/o/collection1/
[junit4:junit4]   2> 12618 T262 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57087/dk_vlc/o
[junit4:junit4]   2> 12618 T262 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 12618 T262 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 12619 T262 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12620 T262 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12622 T204 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12622 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12623 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12812 T204 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12815 T204 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57096
[junit4:junit4]   2> 12816 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12817 T204 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12817 T204 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360188216451
[junit4:junit4]   2> 12818 T204 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360188216451/solr.xml
[junit4:junit4]   2> 12818 T204 oasc.CoreContainer.<init> New CoreContainer 494392306
[junit4:junit4]   2> 12819 T204 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360188216451/'
[junit4:junit4]   2> 12820 T204 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360188216451/'
[junit4:junit4]   2> 12876 T204 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 12876 T204 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12877 T204 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12877 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12878 T204 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12878 T204 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12879 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12879 T204 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12880 T204 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12880 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12903 T204 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12923 T204 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57059/solr
[junit4:junit4]   2> 12924 T204 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12924 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@43797f6
[junit4:junit4]   2> 12925 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12926 T273 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12927 T273 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 12927 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57097
[junit4:junit4]   2> 12927 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57097
[junit4:junit4]   2> 12929 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e3340009 with negotiated timeout 20000 for client /140.211.11.196:57097
[junit4:junit4]   2> 12930 T273 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e3340009, negotiated timeout = 20000
[junit4:junit4]   2> 12930 T274 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43797f6 name:ZooKeeperConnection Watcher:127.0.0.1:57059 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12930 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12932 T209 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cb188e3340009
[junit4:junit4]   2> 12945 T274 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12945 T206 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57097 which had sessionid 0x13cb188e3340009
[junit4:junit4]   2> 12945 T204 oaz.ZooKeeper.close Session: 0x13cb188e3340009 closed
[junit4:junit4]   2> 12945 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12959 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@314adba0
[junit4:junit4]   2> 12960 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12960 T275 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12961 T275 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 12961 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57098
[junit4:junit4]   2> 12961 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57098
[junit4:junit4]   2> 12963 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e334000a with negotiated timeout 20000 for client /140.211.11.196:57098
[junit4:junit4]   2> 12963 T275 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e334000a, negotiated timeout = 20000
[junit4:junit4]   2> 12963 T276 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@314adba0 name:ZooKeeperConnection Watcher:127.0.0.1:57059/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12963 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12964 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e334000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12966 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e334000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12968 T204 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57096_dk_vlc%2Fo
[junit4:junit4]   2> 12969 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e334000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57096_dk_vlc%2Fo Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57096_dk_vlc%2Fo
[junit4:junit4]   2> 12976 T204 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57096_dk_vlc%2Fo
[junit4:junit4]   2> 12978 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12979 T261 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12979 T246 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12979 T261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12979 T233 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12979 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12980 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12981 T227 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12983 T204 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12990 T277 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360188216451/collection1
[junit4:junit4]   2> 12991 T277 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12991 T277 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12991 T277 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12993 T277 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360188216451/collection1/'
[junit4:junit4]   2> 12994 T277 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360188216451/collection1/lib/README' to classloader
[junit4:junit4]   2> 12995 T277 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360188216451/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13049 T277 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 13133 T277 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13233 T277 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13241 T277 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13885 T277 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13897 T277 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13902 T277 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13926 T277 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13931 T277 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13937 T277 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13938 T277 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13940 T277 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360188216451/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/
[junit4:junit4]   2> 13940 T277 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d8307be
[junit4:junit4]   2> 13941 T277 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 13942 T277 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3 forceNew:false
[junit4:junit4]   2> 13942 T277 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3
[junit4:junit4]   2> 13943 T277 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index/
[junit4:junit4]   2> 13943 T277 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13944 T277 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index forceNew:false
[junit4:junit4]   2> 13964 T277 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5bd0873d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13964 T277 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13965 T277 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index
[junit4:junit4]   2> 13966 T277 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13969 T277 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13969 T277 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13969 T277 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13970 T277 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13970 T277 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13971 T277 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13971 T277 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13972 T277 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13973 T277 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13973 T277 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13974 T277 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13975 T277 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13975 T277 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13976 T277 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13976 T277 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13977 T277 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13977 T277 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13978 T277 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13979 T277 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13979 T277 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13980 T277 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13981 T277 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13982 T277 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13982 T277 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13983 T277 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13984 T277 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13996 T277 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14001 T277 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14002 T277 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14005 T277 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3
[junit4:junit4]   2> 14007 T277 oass.SolrIndexSearcher.<init> Opening Searcher@2db0dae3 main
[junit4:junit4]   2> 14008 T277 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14008 T277 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14009 T277 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14016 T277 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14075 T278 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2db0dae3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14083 T277 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14083 T277 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14096 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14097 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57087_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57087/dk_vlc/o"}
[junit4:junit4]   2> 14101 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57096_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57096/dk_vlc/o"}
[junit4:junit4]   2> 14101 T228 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14102 T228 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 14113 T261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14113 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14113 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14113 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14113 T276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15086 T277 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15086 T277 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57096/dk_vlc/o collection:collection1 shard:shard1
[junit4:junit4]   2> 15118 T277 oasc.ZkController.register We are http://127.0.0.1:57096/dk_vlc/o/collection1/ and leader is http://127.0.0.1:57075/dk_vlc/o/collection1/
[junit4:junit4]   2> 15118 T277 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57096/dk_vlc/o
[junit4:junit4]   2> 15118 T277 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15119 T277 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@5bccf8f3 url=http://127.0.0.1:57096/dk_vlc/o/collection1 node=127.0.0.1:57096_dk_vlc%2Fo C4_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57096_dk_vlc%2Fo, base_url=http://127.0.0.1:57096/dk_vlc/o}
[junit4:junit4]   2> 15127 T279 C4 P57096 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15127 T277 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15127 T279 C4 P57096 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15128 T279 C4 P57096 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15128 T279 C4 P57096 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15128 T204 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 15129 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15130 T279 C4 P57096 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15130 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15339 T204 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15342 T204 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57102
[junit4:junit4]   2> 15343 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15344 T204 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15344 T204 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360188218961
[junit4:junit4]   2> 15345 T204 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360188218961/solr.xml
[junit4:junit4]   2> 15345 T204 oasc.CoreContainer.<init> New CoreContainer 712629000
[junit4:junit4]   2> 15346 T204 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360188218961/'
[junit4:junit4]   2> 15346 T204 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360188218961/'
[junit4:junit4]   2> 15416 T204 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 15416 T204 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15417 T204 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15417 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15418 T204 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15418 T204 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15419 T204 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15419 T204 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15420 T204 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15420 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15447 T204 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15470 T204 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57059/solr
[junit4:junit4]   2> 15471 T204 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15471 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3c5edf64
[junit4:junit4]   2> 15473 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15473 T289 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15474 T289 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 15474 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57103
[junit4:junit4]   2> 15475 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57103
[junit4:junit4]   2> 15476 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e334000b with negotiated timeout 20000 for client /140.211.11.196:57103
[junit4:junit4]   2> 15476 T289 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e334000b, negotiated timeout = 20000
[junit4:junit4]   2> 15477 T290 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c5edf64 name:ZooKeeperConnection Watcher:127.0.0.1:57059 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15477 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15479 T209 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cb188e334000b
[junit4:junit4]   2> 15480 T290 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15480 T206 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57103 which had sessionid 0x13cb188e334000b
[junit4:junit4]   2> 15480 T204 oaz.ZooKeeper.close Session: 0x13cb188e334000b closed
[junit4:junit4]   2> 15480 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 15494 T204 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57059/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@977faf
[junit4:junit4]   2> 15495 T204 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15496 T291 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57059. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15496 T291 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57059, initiating session
[junit4:junit4]   2> 15496 T206 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57104
[junit4:junit4]   2> 15497 T206 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57104
[junit4:junit4]   2> 15498 T208 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cb188e334000c with negotiated timeout 20000 for client /140.211.11.196:57104
[junit4:junit4]   2> 15499 T291 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57059, sessionid = 0x13cb188e334000c, negotiated timeout = 20000
[junit4:junit4]   2> 15499 T292 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@977faf name:ZooKeeperConnection Watcher:127.0.0.1:57059/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15499 T204 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15500 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e334000c type:create cxid:0x1 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15502 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e334000c type:create cxid:0x2 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15510 T204 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57102_dk_vlc%2Fo
[junit4:junit4]   2> 15510 T209 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cb188e334000c type:delete cxid:0x4 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57102_dk_vlc%2Fo Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57102_dk_vlc%2Fo
[junit4:junit4]   2> 15512 T204 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57102_dk_vlc%2Fo
[junit4:junit4]   2> 15515 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15515 T261 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15516 T276 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15516 T261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15516 T233 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15516 T246 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15516 T276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15516 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15516 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15518 T227 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15520 T204 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15529 T293 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360188218961/collection1
[junit4:junit4]   2> 15529 T293 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15530 T293 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15530 T293 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15532 T293 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360188218961/collection1/'
[junit4:junit4]   2> 15533 T293 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360188218961/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15534 T293 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360188218961/collection1/lib/README' to classloader
[junit4:junit4]   2> 15629 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15631 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57096_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57096/dk_vlc/o"}
[junit4:junit4]   2> 15634 T293 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 15668 T261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15668 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15669 T276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15669 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15668 T292 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15669 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15714 T293 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 15815 T293 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 15840 T293 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 16631 T293 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16650 T293 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16654 T293 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16675 T293 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16697 T293 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16702 T293 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16703 T293 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16704 T293 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360188218961/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/
[junit4:junit4]   2> 16705 T293 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d8307be
[junit4:junit4]   2> 16705 T293 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 16706 T293 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4 forceNew:false
[junit4:junit4]   2> 16707 T293 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4
[junit4:junit4]   2> 16707 T293 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index/
[junit4:junit4]   2> 16708 T293 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16708 T293 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index forceNew:false
[junit4:junit4]   2> 16714 T293 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@51f0ad2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16715 T293 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 16716 T293 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index
[junit4:junit4]   2> 16716 T293 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16718 T293 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16718 T293 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16718 T293 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16719 T293 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16719 T293 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16719 T293 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16720 T293 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16720 T293 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16730 T293 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16731 T293 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16732 T293 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16732 T293 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16732 T293 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16733 T293 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16733 T293 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16733 T293 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16734 T293 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16734 T293 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16735 T293 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16735 T293 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16736 T293 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16736 T293 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16737 T293 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16737 T293 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16738 T293 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16738 T293 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16747 T293 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16750 T293 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16752 T293 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16754 T293 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4
[junit4:junit4]   2> 16755 T293 oass.SolrIndexSearcher.<init> Opening Searcher@83d069e main
[junit4:junit4]   2> 16756 T293 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16756 T293 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16756 T293 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16770 T293 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 16822 T294 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@83d069e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 16826 T293 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 16827 T293 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17175 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17175 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57102_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57102/dk_vlc/o"}
[junit4:junit4]   2> 17176 T228 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 17176 T228 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 17199 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17199 T292 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17199 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17199 T261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17199 T276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17199 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17829 T293 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 17829 T293 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57102/dk_vlc/o collection:collection1 shard:shard2
[junit4:junit4]   2> 17858 T293 oasc.ZkController.register We are http://127.0.0.1:57102/dk_vlc/o/collection1/ and leader is http://127.0.0.1:57087/dk_vlc/o/collection1/
[junit4:junit4]   2> 17858 T293 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57102/dk_vlc/o
[junit4:junit4]   2> 17858 T293 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 17858 T293 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@49e3a921 url=http://127.0.0.1:57102/dk_vlc/o/collection1 node=127.0.0.1:57102_dk_vlc%2Fo C5_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57102_dk_vlc%2Fo, base_url=http://127.0.0.1:57102/dk_vlc/o}
[junit4:junit4]   2> 17859 T295 C5 P57102 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 17859 T293 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17860 T295 C5 P57102 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 17860 T295 C5 P57102 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 17860 T295 C5 P57102 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17861 T204 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 17861 T204 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 17862 T204 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17864 T295 C5 P57102 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17886 T204 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17888 T204 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 17889 T204 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18706 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18707 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57102_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57102/dk_vlc/o"}
[junit4:junit4]   2>  C4_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57096_dk_vlc%2Fo, base_url=http://127.0.0.1:57096/dk_vlc/o}
[junit4:junit4]   2> 18743 T279 C4 P57096 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57075/dk_vlc/o/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18751 T261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18751 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18751 T292 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18751 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18751 T276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18751 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18755 T279 C4 P57096 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 18776 T279 C4 P57096 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57096/dk_vlc/o START replicas=[http://127.0.0.1:57075/dk_vlc/o/collection1/] nUpdates=100
[junit4:junit4]   2> 18788 T279 C4 P57096 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18789 T279 C4 P57096 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18789 T279 C4 P57096 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18790 T279 C4 P57096 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18790 T279 C4 P57096 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18790 T279 C4 P57096 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57075/dk_vlc/o/collection1/. core=collection1
[junit4:junit4]   2> 18790 T279 C4 P57096 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@5af824d7 url=http://127.0.0.1:57075/dk_vlc/o/collection1 node=127.0.0.1:57075_dk_vlc%2Fo C6_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57075_dk_vlc%2Fo, base_url=http://127.0.0.1:57075/dk_vlc/o, leader=true}
[junit4:junit4]   2> 18845 T238 C6 P57075 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=7 
[junit4:junit4]   2> 18862 T237 C6 P57075 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18863 T237 C6 P57075 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1
[junit4:junit4]   2> 18868 T237 C6 P57075 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d95aab2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18869 T237 C6 P57075 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18873 T237 C6 P57075 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d95aab2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d95aab2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18873 T237 C6 P57075 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18875 T237 C6 P57075 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1
[junit4:junit4]   2> 18875 T237 C6 P57075 oass.SolrIndexSearcher.<init> Opening Searcher@36ab61a8 realtime
[junit4:junit4]   2> 18875 T237 C6 P57075 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18876 T237 C6 P57075 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 35
[junit4:junit4]   2> 18891 T204 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18937 T279 C4 P57096 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18938 T279 C4 P57096 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 18965 T279 C4 P57096 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 19058 T239 C6 P57075 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 19059 T239 C6 P57075 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=91 
[junit4:junit4]   2> 19061 T279 C4 P57096 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 19062 T279 C4 P57096 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 19062 T279 C4 P57096 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 19066 T239 C6 P57075 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1
[junit4:junit4]   2> 19067 T239 C6 P57075 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty1/index
[junit4:junit4]   2> 19068 T239 C6 P57075 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=3 
[junit4:junit4]   2> 19069 T279 C4 P57096 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 19071 T279 C4 P57096 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index.20130206140342882 forceNew:false
[junit4:junit4]   2> 19072 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3
[junit4:junit4]   2> 19072 T279 C4 P57096 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index.20130206140342882 lockFactory=org.apache.lucene.store.NativeFSLockFactory@580cd2ca; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 19091 T239 C6 P57075 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=8 
[junit4:junit4]   2> 19107 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index
[junit4:junit4]   2> 19107 T279 C4 P57096 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 19108 T279 C4 P57096 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130206140342882
[junit4:junit4]   2> 19115 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3
[junit4:junit4]   2> 19116 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3
[junit4:junit4]   2> 19116 T279 C4 P57096 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index.20130206140342882
[junit4:junit4]   2> 19120 T279 C4 P57096 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index.20130206140342882 lockFactory=org.apache.lucene.store.NativeFSLockFactory@580cd2ca; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 19121 T279 C4 P57096 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 19123 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3
[junit4:junit4]   2> 19128 T279 C4 P57096 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 19128 T279 C4 P57096 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 19128 T279 C4 P57096 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 19129 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index.20130206140342882
[junit4:junit4]   2> 19130 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3
[junit4:junit4]   2> 19134 T279 C4 P57096 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index.20130206140342882 lockFactory=org.apache.lucene.store.NativeFSLockFactory@580cd2ca; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 19134 T279 C4 P57096 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 19135 T279 C4 P57096 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 19135 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3
[junit4:junit4]   2> 19136 T279 C4 P57096 oass.SolrIndexSearcher.<init> Opening Searcher@8bbfd7 main
[junit4:junit4]   2> 19138 T278 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8bbfd7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 19138 T278 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index
[junit4:junit4]   2> 19139 T279 C4 P57096 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 19142 T279 C4 P57096 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index.20130206140342882 lockFactory=org.apache.lucene.store.NativeFSLockFactory@580cd2ca; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index.20130206140342882 lockFactory=org.apache.lucene.store.NativeFSLockFactory@580cd2ca; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 19142 T279 C4 P57096 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 19144 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3
[junit4:junit4]   2> 19144 T279 C4 P57096 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 19144 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index.20130206140342882
[junit4:junit4]   2> 19144 T279 C4 P57096 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty3/index
[junit4:junit4]   2> 19145 T279 C4 P57096 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 19145 T279 C4 P57096 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 19145 T279 C4 P57096 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 19145 T279 C4 P57096 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19152 T279 C4 P57096 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19894 T204 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20257 T228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20258 T228 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57096_dk_vlc%2Fo",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57096/dk_vlc/o"}
[junit4:junit4]   2> 20279 T227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20279 T246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20279 T261 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20279 T276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20279 T233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20279 T292 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2>  C5_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57102_dk_vlc%2Fo, base_url=http://127.0.0.1:57102/dk_vlc/o}
[junit4:junit4]   2> 20890 T295 C5 P57102 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57087/dk_vlc/o/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 20890 T295 C5 P57102 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57102/dk_vlc/o START replicas=[http://127.0.0.1:57087/dk_vlc/o/collection1/] nUpdates=100
[junit4:junit4]   2> 20891 T295 C5 P57102 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20891 T295 C5 P57102 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20891 T295 C5 P57102 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20891 T295 C5 P57102 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 20892 T295 C5 P57102 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 20892 T295 C5 P57102 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57087/dk_vlc/o/collection1/. core=collection1
[junit4:junit4]   2> 20892 T295 C5 P57102 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20897 T204 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C7 name=collection1 org.apache.solr.core.SolrCore@4241724c url=http://127.0.0.1:57087/dk_vlc/o/collection1 node=127.0.0.1:57087_dk_vlc%2Fo C7_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57087_dk_vlc%2Fo, base_url=http://127.0.0.1:57087/dk_vlc/o, leader=true}
[junit4:junit4]   2> 20909 T252 C7 P57087 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 20912 T253 C7 P57087 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20913 T253 C7 P57087 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2
[junit4:junit4]   2> 20918 T253 C7 P57087 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7cea9a75; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 20919 T253 C7 P57087 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 20949 T253 C7 P57087 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7cea9a75; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7cea9a75; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20949 T253 C7 P57087 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20950 T253 C7 P57087 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2
[junit4:junit4]   2> 20951 T253 C7 P57087 oass.SolrIndexSearcher.<init> Opening Searcher@4b713040 realtime
[junit4:junit4]   2> 20951 T253 C7 P57087 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20952 T253 C7 P57087 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 40
[junit4:junit4]   2> 20953 T295 C5 P57102 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20953 T295 C5 P57102 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 20956 T254 C7 P57087 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20957 T254 C7 P57087 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 20958 T295 C5 P57102 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 20958 T295 C5 P57102 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 20958 T295 C5 P57102 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 20961 T254 C7 P57087 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2
[junit4:junit4]   2> 20962 T254 C7 P57087 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty2/index
[junit4:junit4]   2> 20962 T254 C7 P57087 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 20963 T295 C5 P57102 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 20965 T295 C5 P57102 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index.20130206140344776 forceNew:false
[junit4:junit4]   2> 20966 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4
[junit4:junit4]   2> 20967 T295 C5 P57102 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index.20130206140344776 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2bf257fd; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 20972 T254 C7 P57087 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
[junit4:junit4]   2> 20974 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index
[junit4:junit4]   2> 20974 T295 C5 P57102 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 20974 T295 C5 P57102 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130206140344776
[junit4:junit4]   2> 20976 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4
[junit4:junit4]   2> 20977 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4
[junit4:junit4]   2> 20977 T295 C5 P57102 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index.20130206140344776
[junit4:junit4]   2> 20982 T295 C5 P57102 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index.20130206140344776 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2bf257fd; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20982 T295 C5 P57102 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20984 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4
[junit4:junit4]   2> 20984 T295 C5 P57102 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 20985 T295 C5 P57102 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 20985 T295 C5 P57102 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 20986 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index.20130206140344776
[junit4:junit4]   2> 20987 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4
[junit4:junit4]   2> 20991 T295 C5 P57102 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index.20130206140344776 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2bf257fd; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20992 T295 C5 P57102 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20992 T295 C5 P57102 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 20993 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4
[junit4:junit4]   2> 20993 T295 C5 P57102 oass.SolrIndexSearcher.<init> Opening Searcher@236c08e1 main
[junit4:junit4]   2> 20996 T294 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@236c08e1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 20997 T294 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index
[junit4:junit4]   2> 20997 T295 C5 P57102 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 21002 T295 C5 P57102 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index.20130206140344776 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2bf257fd; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index.20130206140344776 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2bf257fd; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 21002 T295 C5 P57102 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 21004 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4
[junit4:junit4]   2> 21004 T295 C5 P57102 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21004 T295 C5 P57102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360188203820/jetty4/index.20130206140344776
[junit4:junit4]   2> 21004 T295 

[...truncated too long message...]

MethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 145996 T2479 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=MockRandom), text=PostingsFormat(name=Memory doPackFST= true), _version_=PostingsFormat(name=MockRandom), rnd_b=PostingsFormat(name=Memory doPackFST= false), intDefault=PostingsFormat(name=MockRandom), timestamp=PostingsFormat(name=MockRandom), id=PostingsFormat(name=Memory doPackFST= false), a_t=PostingsFormat(name=MockRandom), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=PostingsFormat(name=TestBloomFilteredLucene41Postings), other_tl1=PostingsFormat(name=MockRandom), multiDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_si=PostingsFormat(name=TestBloomFilteredLucene41Postings)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=it_IT, timezone=Europe/Prague
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=2,free=182819496,total=526385152
[junit4:junit4]   2> NOTE: All tests run in this JVM: [WordBreakSolrSpellCheckerTest, ClusterStateUpdateTest, LeaderElectionIntegrationTest, IndexBasedSpellCheckerTest, TestElisionMultitermQuery, TestGroupingSearch, PluginInfoTest, SystemInfoHandlerTest, BasicDistributedZkTest, FieldAnalysisRequestHandlerTest, TestFiltering, MinimalSchemaTest, TestSolr4Spatial, SpellCheckCollatorTest, CopyFieldTest, DirectUpdateHandlerOptimizeTest, BinaryUpdateRequestHandlerTest, CoreContainerCoreInitFailuresTest, TestQueryTypes, DateMathParserTest, SolrCmdDistributorTest, TestWordDelimiterFilterFactory, TestQuerySenderNoQuery, ResourceLoaderTest, DistanceFunctionTest, CacheHeaderTest, SolrCoreCheckLockOnStartupTest, TestLazyCores, MoreLikeThisHandlerTest, CoreAdminHandlerTest, TestRandomFaceting, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, AlternateDirectoryTest, SearchHandlerTest, TestCoreContainer, TestComponentsName, TestFastLRUCache, CurrencyFieldTest, JsonLoaderTest, DefaultValueUpdateProcessorTest, SuggesterWFSTTest, DirectSolrConnectionTest, SolrPluginUtilsTest, TestConfig, CircularListTest, NoCacheHeaderTest, NotRequiredUniqueKeyTest, TestDocumentBuilder, TestPerFieldSimilarity, QueryElevationComponentTest, TestPartialUpdateDeduplication, TestBinaryResponseWriter, ChaosMonkeyNothingIsSafeTest, SimpleFacetsTest, TestCollationField, TestAtomicUpdateErrorCases, JSONWriterTest, ZkControllerTest, ZkCLITest, TermsComponentTest, DistributedSpellCheckComponentTest, TestExtendedDismaxParser, SoftAutoCommitTest, AutoCommitTest, BasicZkTest, DistributedTermsComponentTest, TestRecovery, SolrCoreTest, TestRangeQuery, LukeRequestHandlerTest, TestSort, FileBasedSpellCheckerTest, ConvertedLegacyTest, SortByFunctionTest, SampleTest, TestIndexSearcher, SolrRequestParserTest, TestCSVLoader, PolyFieldTest, SpatialFilterTest, MBeansHandlerTest, TestPseudoReturnFields, IndexSchemaTest, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, StandardRequestHandlerTest, TestPHPSerializedResponseWriter, XmlUpdateRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TestSolrQueryParser, FastVectorHighlighterTest, TestReversedWildcardFilterFactory, DebugComponentTest, RequestHandlersTest, TestOmitPositions, SuggesterFSTTest, TestArbitraryIndexDir, LoggingHandlerTest, UniqFieldsUpdateProcessorFactoryTest, UpdateParamsTest, CSVRequestHandlerTest, TestSearchPerf, TestSolrDeletionPolicy1, TestLuceneMatchVersion, UpdateRequestProcessorFactoryTest, FieldMutatingUpdateProcessorTest, TestBinaryField, TestSolrDeletionPolicy2, TestPropInjectDefaults, TestMergePolicyConfig, SolrIndexConfigTest, NumericFieldsTest, TestNumberUtils, TestJmxMonitoredMap, EchoParamsTest, TestPhraseSuggestions, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, PrimUtilsTest, TestSolrXMLSerializer, SpellingQueryConverterTest, TestBadConfig, RAMDirectoryFactoryTest, TestUtils, PreAnalyzedFieldTest, ZkNodePropsTest, AnalysisAfterCoreReloadTest, PathHierarchyTokenizerFactoryTest, TestCharFilters, ClusterStateTest, CollectionsAPIDistributedZkTest, ShardRoutingTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 146.03s, 1 test, 1 failure <<< FAILURES!

[...truncated 59 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 error, 1 failure, 22 ignored (4 assumptions)

Total time: 50 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure