You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/06/30 12:53:58 UTC
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4100 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4100/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([DC0881C3A2CB615A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([DC0881C3A2CB615A]:0)
Build Log:
[...truncated 9509 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 780741 T1699 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 780746 T1699 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372589296941
[junit4:junit4] 2> 780748 T1699 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 780749 T1700 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 780850 T1699 oasc.ZkTestServer.run start zk server on port:51211
[junit4:junit4] 2> 780851 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 780855 T1706 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2195d9e0 name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 780855 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 780856 T1699 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 780874 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 780876 T1708 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c88d5ab name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 780876 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 780877 T1699 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 780887 T1699 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 780889 T1699 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 780892 T1699 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 780895 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 780895 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 780901 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 780902 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 781006 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 781007 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 781010 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 781011 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 781021 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 781021 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 781025 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 781026 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 781029 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 781029 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 781033 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 781033 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 781036 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 781037 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 781306 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 781310 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62307
[junit4:junit4] 2> 781311 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 781311 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 781312 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234
[junit4:junit4] 2> 781312 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/solr.xml
[junit4:junit4] 2> 781313 T1699 oasc.CoreContainer.<init> New CoreContainer 205130362
[junit4:junit4] 2> 781314 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/'
[junit4:junit4] 2> 781314 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/'
[junit4:junit4] 2> 781446 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 781447 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 781448 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 781448 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 781449 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 781449 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 781450 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 781450 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 781451 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 781451 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 781468 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 781469 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 781469 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 781470 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 781473 T1719 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7da2c048 name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 781474 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 781484 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 781495 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 781498 T1721 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@401512e name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 781498 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 781501 T1699 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 781510 T1699 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 781513 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 781520 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62307_
[junit4:junit4] 2> 781522 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62307_
[junit4:junit4] 2> 781525 T1699 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 781542 T1699 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 781544 T1699 oasc.Overseer.start Overseer (id=89954012164784131-127.0.0.1:62307_-n_0000000000) starting
[junit4:junit4] 2> 781554 T1699 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 781565 T1723 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 781566 T1699 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 781569 T1699 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 781576 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 781578 T1722 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 781583 T1724 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 781583 T1724 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 781584 T1724 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 783083 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 783083 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62307_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62307"}
[junit4:junit4] 2> 783084 T1722 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 783084 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 783101 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 783587 T1724 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 783587 T1724 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1
[junit4:junit4] 2> 783587 T1724 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 783588 T1724 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 783588 T1724 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 783590 T1724 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1/'
[junit4:junit4] 2> 783591 T1724 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1/lib/README' to classloader
[junit4:junit4] 2> 783592 T1724 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 783646 T1724 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 783708 T1724 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 783809 T1724 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 783816 T1724 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 784446 T1724 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 784460 T1724 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 784464 T1724 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 784482 T1724 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 784488 T1724 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 784492 T1724 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 784494 T1724 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 784494 T1724 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 784494 T1724 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 784495 T1724 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 784496 T1724 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 784496 T1724 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 784496 T1724 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data/
[junit4:junit4] 2> 784497 T1724 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 784498 T1724 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data
[junit4:junit4] 2> 784498 T1724 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data/index/
[junit4:junit4] 2> 784498 T1724 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 784499 T1724 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data/index
[junit4:junit4] 2> 784503 T1724 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 784504 T1724 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 784507 T1724 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 784507 T1724 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 784508 T1724 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 784509 T1724 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 784509 T1724 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 784509 T1724 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 784510 T1724 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 784510 T1724 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 784511 T1724 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 784526 T1724 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 784533 T1724 oass.SolrIndexSearcher.<init> Opening Searcher@12124b16 main
[junit4:junit4] 2> 784534 T1724 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 784534 T1724 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 784538 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12124b16 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 784541 T1724 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 784541 T1724 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62307 collection:control_collection shard:shard1
[junit4:junit4] 2> 784542 T1724 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 784561 T1724 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 784564 T1724 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 784564 T1724 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 784565 T1724 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:62307/collection1/
[junit4:junit4] 2> 784565 T1724 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 784565 T1724 oasc.SyncStrategy.syncToMe http://127.0.0.1:62307/collection1/ has no replicas
[junit4:junit4] 2> 784565 T1724 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:62307/collection1/ shard1
[junit4:junit4] 2> 784566 T1724 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 784605 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 784623 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 784675 T1724 oasc.ZkController.register We are http://127.0.0.1:62307/collection1/ and leader is http://127.0.0.1:62307/collection1/
[junit4:junit4] 2> 784675 T1724 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62307
[junit4:junit4] 2> 784675 T1724 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 784675 T1724 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 784676 T1724 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 784677 T1724 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 784679 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 784679 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 784680 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 784696 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 784698 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 784700 T1728 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d4ae61f name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 784700 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 784702 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 784704 T1699 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 784978 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 784980 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10217
[junit4:junit4] 2> 784981 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 784982 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 784982 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898
[junit4:junit4] 2> 784983 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/solr.xml
[junit4:junit4] 2> 784983 T1699 oasc.CoreContainer.<init> New CoreContainer 110884964
[junit4:junit4] 2> 784984 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/'
[junit4:junit4] 2> 784984 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/'
[junit4:junit4] 2> 785117 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 785118 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 785118 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 785119 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 785119 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 785120 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 785120 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 785121 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 785121 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 785122 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 785139 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 785140 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 785141 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 785142 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 785145 T1739 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c706b4d name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 785145 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 785164 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 785177 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 785179 T1741 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cf4bdd9 name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 785179 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 785190 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 786130 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 786131 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62307_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62307"}
[junit4:junit4] 2> 786152 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786152 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786152 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786194 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10217_
[junit4:junit4] 2> 786196 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10217_
[junit4:junit4] 2> 786199 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786199 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786199 T1741 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 786200 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 786201 T1721 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 786211 T1728 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 786217 T1742 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 786217 T1742 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 786218 T1742 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 787657 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 787658 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10217_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10217"}
[junit4:junit4] 2> 787659 T1722 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 787659 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 787675 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 787675 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 787675 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 788221 T1742 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 788221 T1742 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1
[junit4:junit4] 2> 788221 T1742 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 788222 T1742 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 788222 T1742 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 788224 T1742 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1/'
[junit4:junit4] 2> 788225 T1742 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 788226 T1742 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1/lib/README' to classloader
[junit4:junit4] 2> 788280 T1742 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 788342 T1742 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 788444 T1742 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 788452 T1742 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 789074 T1742 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 789088 T1742 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 789091 T1742 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 789109 T1742 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 789114 T1742 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 789118 T1742 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 789119 T1742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 789120 T1742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 789120 T1742 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 789121 T1742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 789121 T1742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 789122 T1742 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 789122 T1742 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1/
[junit4:junit4] 2> 789122 T1742 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 789123 T1742 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1
[junit4:junit4] 2> 789124 T1742 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1/index/
[junit4:junit4] 2> 789124 T1742 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 789124 T1742 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1/index
[junit4:junit4] 2> 789128 T1742 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> 789128 T1742 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 789132 T1742 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 789132 T1742 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 789133 T1742 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 789134 T1742 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 789134 T1742 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 789134 T1742 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 789135 T1742 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 789136 T1742 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 789136 T1742 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 789151 T1742 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 789158 T1742 oass.SolrIndexSearcher.<init> Opening Searcher@4179ae1f main
[junit4:junit4] 2> 789159 T1742 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 789160 T1742 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 789165 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4179ae1f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 789167 T1742 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 789167 T1742 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10217 collection:collection1 shard:shard1
[junit4:junit4] 2> 789168 T1742 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 789186 T1742 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 789189 T1742 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 789189 T1742 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 789190 T1742 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10217/collection1/
[junit4:junit4] 2> 789190 T1742 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 789190 T1742 oasc.SyncStrategy.syncToMe http://127.0.0.1:10217/collection1/ has no replicas
[junit4:junit4] 2> 789191 T1742 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10217/collection1/ shard1
[junit4:junit4] 2> 789191 T1742 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 790688 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 790785 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 790785 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 790785 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 790829 T1742 oasc.ZkController.register We are http://127.0.0.1:10217/collection1/ and leader is http://127.0.0.1:10217/collection1/
[junit4:junit4] 2> 790829 T1742 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10217
[junit4:junit4] 2> 790829 T1742 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 790829 T1742 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 790830 T1742 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 790866 T1742 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 790867 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 790868 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 790869 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 791151 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 791154 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44467
[junit4:junit4] 2> 791155 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 791155 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 791156 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078
[junit4:junit4] 2> 791156 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/solr.xml
[junit4:junit4] 2> 791157 T1699 oasc.CoreContainer.<init> New CoreContainer 1371466089
[junit4:junit4] 2> 791158 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/'
[junit4:junit4] 2> 791158 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/'
[junit4:junit4] 2> 791290 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 791291 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 791291 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 791292 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 791292 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 791293 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 791293 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 791294 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 791294 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 791295 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 791311 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 791312 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 791312 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 791313 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 791317 T1755 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e5dbba1 name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 791317 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 791334 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 791345 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 791347 T1757 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7344adea name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 791347 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 791359 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 792322 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 792323 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10217_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10217"}
[junit4:junit4] 2> 792340 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792340 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792340 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792340 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792363 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44467_
[junit4:junit4] 2> 792365 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44467_
[junit4:junit4] 2> 792368 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792368 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792368 T1741 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 792368 T1757 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 792368 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 792369 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 792369 T1728 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 792370 T1721 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 792376 T1758 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 792377 T1758 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 792378 T1758 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 793845 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 793846 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44467_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44467"}
[junit4:junit4] 2> 793847 T1722 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 793847 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 793862 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 793862 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 793862 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 793862 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 794381 T1758 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 794381 T1758 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1
[junit4:junit4] 2> 794381 T1758 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 794382 T1758 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 794382 T1758 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 794384 T1758 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1/'
[junit4:junit4] 2> 794386 T1758 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1/lib/README' to classloader
[junit4:junit4] 2> 794386 T1758 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 794440 T1758 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 794502 T1758 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 794604 T1758 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 794611 T1758 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 795239 T1758 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 795254 T1758 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 795258 T1758 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 795276 T1758 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 795281 T1758 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 795285 T1758 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 795287 T1758 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 795287 T1758 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 795287 T1758 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 795288 T1758 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 795288 T1758 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 795289 T1758 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 795289 T1758 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2/
[junit4:junit4] 2> 795290 T1758 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 795291 T1758 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2
[junit4:junit4] 2> 795291 T1758 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2/index/
[junit4:junit4] 2> 795292 T1758 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 795292 T1758 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2/index
[junit4:junit4] 2> 795296 T1758 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@24605c34 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1849901e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 795296 T1758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 795300 T1758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 795300 T1758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 795301 T1758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 795302 T1758 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 795303 T1758 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 795303 T1758 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 795303 T1758 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 795304 T1758 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 795305 T1758 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 795321 T1758 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 795328 T1758 oass.SolrIndexSearcher.<init> Opening Searcher@312f8a90 main
[junit4:junit4] 2> 795330 T1758 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 795330 T1758 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 795336 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@312f8a90 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 795339 T1758 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 795339 T1758 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44467 collection:collection1 shard:shard2
[junit4:junit4] 2> 795340 T1758 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 795356 T1758 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 795359 T1758 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 795359 T1758 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 795360 T1758 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44467/collection1/
[junit4:junit4] 2> 795360 T1758 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 795360 T1758 oasc.SyncStrategy.syncToMe http://127.0.0.1:44467/collection1/ has no replicas
[junit4:junit4] 2> 795361 T1758 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44467/collection1/ shard2
[junit4:junit4] 2> 795361 T1758 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 795367 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 795374 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 795374 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 795374 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 795374 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 795418 T1758 oasc.ZkController.register We are http://127.0.0.1:44467/collection1/ and leader is http://127.0.0.1:44467/collection1/
[junit4:junit4] 2> 795418 T1758 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44467
[junit4:junit4] 2> 795418 T1758 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 795419 T1758 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 795419 T1758 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 795441 T1758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 795443 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 795443 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 795444 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 795733 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 795735 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:11586
[junit4:junit4] 2> 795736 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 795737 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 795737 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655
[junit4:junit4] 2> 795738 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/solr.xml
[junit4:junit4] 2> 795738 T1699 oasc.CoreContainer.<init> New CoreContainer 1634567671
[junit4:junit4] 2> 795739 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/'
[junit4:junit4] 2> 795740 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/'
[junit4:junit4] 2> 795878 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 795879 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 795879 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 795880 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 795880 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 795881 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 795881 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 795882 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 795882 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 795883 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 795901 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 795902 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 795902 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 795903 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 795906 T1771 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f79fbdb name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 795907 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 795949 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 795961 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 795964 T1773 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70ef8558 name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 795964 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 795977 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 796880 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 796881 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44467_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44467"}
[junit4:junit4] 2> 796902 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796902 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796902 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796902 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796902 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796980 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11586_
[junit4:junit4] 2> 796982 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11586_
[junit4:junit4] 2> 796984 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796984 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796985 T1757 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796985 T1773 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796985 T1741 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796985 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 796985 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 796985 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 796986 T1721 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796986 T1728 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796999 T1774 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 796999 T1774 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 797001 T1774 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 798408 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 798409 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:11586_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11586"}
[junit4:junit4] 2> 798410 T1722 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 798410 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 798427 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 798427 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 798427 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 798428 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 798428 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 799003 T1774 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 799003 T1774 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1
[junit4:junit4] 2> 799003 T1774 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 799004 T1774 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 799004 T1774 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 799006 T1774 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1/'
[junit4:junit4] 2> 799007 T1774 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1/lib/README' to classloader
[junit4:junit4] 2> 799008 T1774 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 799062 T1774 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 799124 T1774 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 799225 T1774 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 799233 T1774 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 799861 T1774 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 799875 T1774 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 799879 T1774 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 799897 T1774 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 799903 T1774 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 799908 T1774 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 799909 T1774 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 799909 T1774 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 799909 T1774 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 799910 T1774 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 799911 T1774 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 799911 T1774 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 799911 T1774 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3/
[junit4:junit4] 2> 799912 T1774 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 799913 T1774 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3
[junit4:junit4] 2> 799913 T1774 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3/index/
[junit4:junit4] 2> 799914 T1774 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 799914 T1774 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3/index
[junit4:junit4] 2> 799918 T1774 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_1,generation=1}
[junit4:junit4] 2> 799918 T1774 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 799922 T1774 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 799922 T1774 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 799923 T1774 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 799924 T1774 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 799925 T1774 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 799925 T1774 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 799925 T1774 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 799926 T1774 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 799927 T1774 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 799943 T1774 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 799950 T1774 oass.SolrIndexSearcher.<init> Opening Searcher@6c5dfbd3 main
[junit4:junit4] 2> 799952 T1774 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 799952 T1774 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 799957 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c5dfbd3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 799960 T1774 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 799961 T1774 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:11586 collection:collection1 shard:shard1
[junit4:junit4] 2> 799964 T1774 oasc.ZkController.register We are http://127.0.0.1:11586/collection1/ and leader is http://127.0.0.1:10217/collection1/
[junit4:junit4] 2> 799964 T1774 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:11586
[junit4:junit4] 2> 799965 T1774 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 799965 T1774 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C335 name=collection1 org.apache.solr.core.SolrCore@4083bcde url=http://127.0.0.1:11586/collection1 node=127.0.0.1:11586_ C335_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:11586_, base_url=http://127.0.0.1:11586}
[junit4:junit4] 2> 799965 T1776 C335 P11586 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 799966 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 799966 T1774 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 799966 T1776 C335 P11586 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 799967 T1776 C335 P11586 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 799967 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 799968 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 799968 T1776 C335 P11586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 799969 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 799995 T1731 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 800261 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 800264 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57178
[junit4:junit4] 2> 800264 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 800265 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 800265 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187
[junit4:junit4] 2> 800266 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/solr.xml
[junit4:junit4] 2> 800266 T1699 oasc.CoreContainer.<init> New CoreContainer 1357457359
[junit4:junit4] 2> 800267 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/'
[junit4:junit4] 2> 800268 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/'
[junit4:junit4] 2> 800402 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 800403 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 800404 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 800404 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 800405 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 800405 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 800406 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 800406 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 800407 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 800407 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 800425 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 800426 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 800426 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 800428 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 800455 T1788 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6cdb72a5 name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 800456 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 800490 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 800503 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 800508 T1790 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c6a43d name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 800508 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 800531 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 801434 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 801435 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:11586_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11586"}
[junit4:junit4] 2> 801458 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801535 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57178_
[junit4:junit4] 2> 801537 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57178_
[junit4:junit4] 2> 801539 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801540 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801540 T1757 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801540 T1773 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801540 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801540 T1790 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801540 T1741 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801541 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801541 T1721 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801541 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801541 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801554 T1728 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801560 T1791 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 801560 T1791 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 801562 T1791 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 801998 T1731 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 801998 T1731 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:11586_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 802964 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 802965 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57178_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57178"}
[junit4:junit4] 2> 802965 T1722 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 802965 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 802986 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 803564 T1791 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 803564 T1791 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1
[junit4:junit4] 2> 803564 T1791 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 803565 T1791 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 803566 T1791 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 803567 T1791 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1/'
[junit4:junit4] 2> 803569 T1791 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 803569 T1791 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1/lib/README' to classloader
[junit4:junit4] 2> 803628 T1791 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 803697 T1791 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 803799 T1791 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 803807 T1791 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> C335_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:11586_, base_url=http://127.0.0.1:11586}
[junit4:junit4] 2> 804000 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:10217/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 804000 T1776 C335 P11586 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:11586 START replicas=[http://127.0.0.1:10217/collection1/] nUpdates=100
[junit4:junit4] 2> 804000 T1776 C335 P11586 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:10217/collection1/. core=collection1
[junit4:junit4] 2> 804002 T1776 C335 P11586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C336 name=collection1 org.apache.solr.core.SolrCore@1a24f684 url=http://127.0.0.1:10217/collection1 node=127.0.0.1:10217_ C336_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:10217_, base_url=http://127.0.0.1:10217, leader=true}
[junit4:junit4] 2> 804008 T1732 C336 P10217 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 804011 T1733 C336 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804013 T1733 C336 P10217 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> 804014 T1733 C336 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 804014 T1733 C336 P10217 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 804014 T1733 C336 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@dcc229d realtime
[junit4:junit4] 2> 804015 T1733 C336 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804015 T1733 C336 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 804016 T1776 C335 P11586 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 804016 T1776 C335 P11586 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 804018 T1734 C336 P10217 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 804018 T1734 C336 P10217 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 804019 T1776 C335 P11586 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 804019 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 804019 T1776 C335 P11586 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 804019 T1776 C335 P11586 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 804021 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 804331 T1791 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 804341 T1791 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 804344 T1791 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 804357 T1791 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 804361 T1791 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 804365 T1791 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 804366 T1791 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 804366 T1791 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 804367 T1791 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 804367 T1791 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 804368 T1791 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 804368 T1791 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 804368 T1791 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4/
[junit4:junit4] 2> 804368 T1791 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 804369 T1791 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4
[junit4:junit4] 2> 804369 T1791 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4/index/
[junit4:junit4] 2> 804370 T1791 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 804370 T1791 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4/index
[junit4:junit4] 2> 804373 T1791 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@118191ef lockFactory=org.apache.lucene.store.NativeFSLockFactory@63f493ec),segFN=segments_1,generation=1}
[junit4:junit4] 2> 804373 T1791 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 804376 T1791 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 804376 T1791 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 804377 T1791 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 804377 T1791 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 804378 T1791 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 804378 T1791 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 804379 T1791 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 804379 T1791 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 804379 T1791 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 804391 T1791 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 804396 T1791 oass.SolrIndexSearcher.<init> Opening Searcher@59e1bafe main
[junit4:junit4] 2> 804397 T1791 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 804398 T1791 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 804402 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59e1bafe main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 804404 T1791 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 804404 T1791 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57178 collection:collection1 shard:shard2
[junit4:junit4] 2> 804407 T1791 oasc.ZkController.register We are http://127.0.0.1:57178/collection1/ and leader is http://127.0.0.1:44467/collection1/
[junit4:junit4] 2> 804407 T1791 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57178
[junit4:junit4] 2> 804407 T1791 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 804408 T1791 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C337 name=collection1 org.apache.solr.core.SolrCore@24b9a4f1 url=http://127.0.0.1:57178/collection1 node=127.0.0.1:57178_ C337_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:57178_, base_url=http://127.0.0.1:57178}
[junit4:junit4] 2> 804408 T1794 C337 P57178 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 804409 T1791 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804409 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 804409 T1794 C337 P57178 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 804409 T1794 C337 P57178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 804410 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 804410 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 804411 T1794 C337 P57178 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 804411 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 804424 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804425 T1747 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 804425 T1699 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 804426 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804491 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804492 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:11586_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11586"}
[junit4:junit4] 2> 804496 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57178_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57178"}
[junit4:junit4] 2> 804508 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 805426 T1747 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 805426 T1747 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:57178_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 805428 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 806430 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C337_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:57178_, base_url=http://127.0.0.1:57178}
[junit4:junit4] 2> 807428 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44467/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 807428 T1794 C337 P57178 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57178 START replicas=[http://127.0.0.1:44467/collection1/] nUpdates=100
[junit4:junit4] 2> 807429 T1794 C337 P57178 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 807429 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 807429 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 807429 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 807430 T1794 C337 P57178 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 807430 T1794 C337 P57178 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:44467/collection1/. core=collection1
[junit4:junit4] 2> 807430 T1794 C337 P57178 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 807432 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C338 name=collection1 org.apache.solr.core.SolrCore@12f10775 url=http://127.0.0.1:44467/collection1 node=127.0.0.1:44467_ C338_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:44467_, base_url=http://127.0.0.1:44467, leader=true}
[junit4:junit4] 2> 807437 T1748 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 807443 T1749 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 807446 T1749 C338 P44467 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@24605c34 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1849901e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 807447 T1749 C338 P44467 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 807447 T1749 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 807448 T1749 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@6dab7184 realtime
[junit4:junit4] 2> 807448 T1749 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 807448 T1749 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 807449 T1794 C337 P57178 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 807450 T1794 C337 P57178 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 807452 T1750 C338 P44467 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 807452 T1750 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 807453 T1794 C337 P57178 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 807453 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 807453 T1794 C337 P57178 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 807453 T1794 C337 P57178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 807455 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 807521 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 807522 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57178_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57178"}
[junit4:junit4] 2> 807539 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807539 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807540 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807540 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807539 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807539 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 808434 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808435 T1699 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C339 name=collection1 org.apache.solr.core.SolrCore@999168f url=http://127.0.0.1:62307/collection1 node=127.0.0.1:62307_ C339_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:62307_, base_url=http://127.0.0.1:62307, leader=true}
[junit4:junit4] 2> 808446 T1711 C339 P62307 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 808447 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 808448 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@1c9aa4d4 realtime
[junit4:junit4] 2> 808449 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439264223669518336)} 0 6
[junit4:junit4] 2> ASYNC NEW_CORE C340 name=collection1 org.apache.solr.core.SolrCore@4083bcde url=http://127.0.0.1:11586/collection1 node=127.0.0.1:11586_ C340_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:11586_, base_url=http://127.0.0.1:11586}
[junit4:junit4] 2> 808472 T1763 C340 P11586 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_1,generation=1}
[junit4:junit4] 2> C337_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:57178_, base_url=http://127.0.0.1:57178}
[junit4:junit4] 2> 808472 T1780 C337 P57178 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@118191ef lockFactory=org.apache.lucene.store.NativeFSLockFactory@63f493ec),segFN=segments_1,generation=1}
[junit4:junit4] 2> 808472 T1763 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 808473 T1780 C337 P57178 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 808475 T1763 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@3b414d20 realtime
[junit4:junit4] 2> 808475 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439264223688392704&update.from=http://127.0.0.1:10217/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439264223688392704)} 0 6
[junit4:junit4] 2> 808476 T1780 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@147e313a realtime
[junit4:junit4] 2> ASYNC NEW_CORE C341 name=collection1 org.apache.solr.core.SolrCore@1a24f684 url=http://127.0.0.1:10217/collection1 node=127.0.0.1:10217_ C341_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:10217_, base_url=http://127.0.0.1:10217, leader=true}
[junit4:junit4] 2> 808476 T1735 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439264223688392704)} 0 15
[junit4:junit4] 2> 808476 T1780 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439264223685246976&update.from=http://127.0.0.1:44467/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439264223685246976)} 0 7
[junit4:junit4] 2> 808477 T1751 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439264223685246976)} 0 19
[junit4:junit4] 2> 808478 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808480 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808483 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808484 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808485 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808487 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808489 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808492 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808495 T1711 C339 P62307 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_2,generation=2}
[junit4:junit4] 2> 808496 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 808497 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@6d5f3409 main
[junit4:junit4] 2> 808498 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808498 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d5f3409 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 808499 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264223719849984)],commit=} 0 8
[junit4:junit4] 2> 808506 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10217/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439264223730335744)]} 0 1
[junit4:junit4] 2> 808506 T1736 C341 P10217 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10217/collection1/, StdNode: http://127.0.0.1:11586/collection1/, StdNode: http://127.0.0.1:44467/collection1/, StdNode: http://127.0.0.1:57178/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 808508 T1737 C341 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808509 T1753 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808509 T1782 C337 P57178 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808509 T1753 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808509 T1763 C340 P11586 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808509 T1782 C337 P57178 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808510 T1753 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@7ed9ef94 main
[junit4:junit4] 2> 808510 T1782 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@60dd3091 main
[junit4:junit4] 2> 808511 T1753 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808512 T1782 C337 P57178 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808512 T1759 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:305)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 808512 T1737 C341 P10217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_2,generation=2}
[junit4:junit4] 2> 808512 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60dd3091 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808513 T1737 C341 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 808514 T1782 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 808514 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7ed9ef94 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808515 T1753 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 808515 T1737 C341 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@3d464b62 main
[junit4:junit4] 2> 808515 T1763 C340 P11586 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_2,generation=2}
[junit4:junit4] 2> 808516 T1737 C341 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808516 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d464b62 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 808516 T1763 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 808517 T1737 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 808519 T1763 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@2c09709a main
[junit4:junit4] 2> 808519 T1763 C340 P11586 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808520 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c09709a main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 808520 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 808521 T1736 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264223730335744)],commit=} 0 20
[junit4:junit4] 2> 808522 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808524 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808528 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808530 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808532 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808535 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808538 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808539 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808540 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808546 T1747 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324736&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808546 T1731 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324736&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808549 T1731 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324736&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808550 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808555 T1749 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324746&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808555 T1733 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324746&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808559 T1733 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324746&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808560 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808561 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808567 T1809 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324757&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808567 T1810 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324757&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808570 T1810 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324757&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808571 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808577 T1814 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324767&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808577 T1813 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324767&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808580 T1813 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324767&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808581 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808586 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808589 T1711 C339 P62307 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_3,generation=3}
[junit4:junit4] 2> 808590 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 808591 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@18e0af4d main
[junit4:junit4] 2> 808592 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808593 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18e0af4d main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 808594 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264223817367552)],commit=} 0 10
[junit4:junit4] 2> 808604 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10217/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439264223832047616)]} 0 2
[junit4:junit4] 2> 808605 T1735 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44467/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439264223832047616)]} 0 7
[junit4:junit4] 2> 808606 T1752 C338 P44467 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10217/collection1/, StdNode: http://127.0.0.1:11586/collection1/, StdNode: http://127.0.0.1:44467/collection1/, StdNode: http://127.0.0.1:57178/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 808608 T1765 C340 P11586 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808608 T1817 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808610 T1780 C337 P57178 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808610 T1735 C341 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808610 T1817 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808611 T1780 C337 P57178 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808612 T1817 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@4a850db5 main
[junit4:junit4] 2> 808612 T1780 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@71c4cab0 main
[junit4:junit4] 2> 808612 T1817 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808613 T1780 C337 P57178 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808613 T1793 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:305)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 808613 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4a850db5 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808614 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@71c4cab0 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808615 T1817 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 808615 T1780 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 808615 T1765 C340 P11586 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_3,generation=3}
[junit4:junit4] 2> 808616 T1765 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 808616 T1735 C341 P10217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_3,generation=3}
[junit4:junit4] 2> 808617 T1735 C341 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 808618 T1765 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@2995be78 main
[junit4:junit4] 2> 808620 T1765 C340 P11586 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808620 T1735 C341 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@5e70b175 main
[junit4:junit4] 2> 808621 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2995be78 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 808621 T1735 C341 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808622 T1765 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 808622 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5e70b175 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 808623 T1735 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 808623 T1752 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 27
[junit4:junit4] 2> 808624 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808626 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808630 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808632 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808633 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808636 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808639 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808640 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808642 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808647 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324838&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808648 T1747 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324838&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808651 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324838&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808652 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808657 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324848&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808658 T1749 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324848&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808661 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324848&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808662 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808663 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808668 T1768 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324859&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808669 T1809 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324859&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808672 T1768 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324859&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808673 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808678 T1769 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324869&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808679 T1814 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324869&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808682 T1769 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324869&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808683 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808688 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808691 T1711 C339 P62307 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_4,generation=4}
[junit4:junit4] 2> 808692 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 808693 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@775d6a2 main
[junit4:junit4] 2> 808694 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808695 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@775d6a2 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 808695 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264223924322304)],commit=} 0 9
[junit4:junit4] 2> 808708 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10217/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439264223940050944)]} 0 2
[junit4:junit4] 2> 808709 T1819 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11586/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439264223940050944)]} 0 8
[junit4:junit4] 2> 808710 T1764 C340 P11586 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10217/collection1/, StdNode: http://127.0.0.1:11586/collection1/, StdNode: http://127.0.0.1:44467/collection1/, StdNode: http://127.0.0.1:57178/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 808713 T1819 C341 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808713 T1783 C337 P57178 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808713 T1823 C340 P11586 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808713 T1824 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808714 T1783 C337 P57178 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808716 T1824 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808716 T1783 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@1ae8b31f main
[junit4:junit4] 2> 808716 T1824 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@4af2ae4d main
[junit4:junit4] 2> 808717 T1783 C337 P57178 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808717 T1824 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808718 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ae8b31f main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808718 T1819 C341 P10217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_4,generation=4}
[junit4:junit4] 2> 808718 T1783 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 808718 T1819 C341 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 808718 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4af2ae4d main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808720 T1824 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 808720 T1823 C340 P11586 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_4,generation=4}
[junit4:junit4] 2> 808720 T1819 C341 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@4d5b490b main
[junit4:junit4] 2> 808720 T1823 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 808721 T1819 C341 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808722 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d5b490b main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 808722 T1819 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 808723 T1823 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@616388e6 main
[junit4:junit4] 2> 808725 T1823 C340 P11586 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808726 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@616388e6 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 808726 T1823 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 808727 T1764 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 29
[junit4:junit4] 2> 808728 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808729 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808733 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808737 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808738 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808741 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808744 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808745 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808746 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808752 T1747 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324942&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808753 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324942&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808755 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324942&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808756 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808761 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324952&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808761 T1749 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324952&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808764 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324952&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808765 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=6
[junit4:junit4] 2> 808766 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808771 T1809 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324962&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808772 T1768 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324962&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808775 T1768 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324962&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808776 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808781 T1814 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324972&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808781 T1769 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324972&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808784 T1769 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324972&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808785 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=6
[junit4:junit4] 2> 808790 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808793 T1711 C339 P62307 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_5,generation=5}
[junit4:junit4] 2> 808794 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 808795 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@2c33afe0 main
[junit4:junit4] 2> 808796 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808797 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c33afe0 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 808797 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264224031277056)],commit=} 0 9
[junit4:junit4] 2> 808810 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10217/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439264224047005696)]} 0 2
[junit4:junit4] 2> 808810 T1826 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57178/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439264224047005696)]} 0 7
[junit4:junit4] 2> 808811 T1781 C337 P57178 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10217/collection1/, StdNode: http://127.0.0.1:11586/collection1/, StdNode: http://127.0.0.1:44467/collection1/, StdNode: http://127.0.0.1:57178/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 808814 T1830 C340 P11586 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808814 T1826 C341 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808814 T1831 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808814 T1784 C337 P57178 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808815 T1831 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808815 T1784 C337 P57178 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808816 T1831 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@b9f002f main
[junit4:junit4] 2> 808816 T1784 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@7847a8fd main
[junit4:junit4] 2> 808817 T1831 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808817 T1784 C337 P57178 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808818 T1830 C340 P11586 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_5,generation=5}
[junit4:junit4] 2> 808818 T1793 oasc.JmxMonitoredMap.put WARN Failed to register info bean: fieldValueCache javax.management.InstanceAlreadyExistsException: solr/collection1:type=fieldValueCache,id=org.apache.solr.search.FastLRUCache
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:309)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 808818 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b9f002f main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808819 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7847a8fd main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808818 T1830 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 808819 T1784 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 808819 T1831 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 808819 T1826 C341 P10217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_5,generation=5}
[junit4:junit4] 2> 808820 T1830 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@19b09c17 main
[junit4:junit4] 2> 808821 T1826 C341 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 808821 T1830 C340 P11586 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808822 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19b09c17 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 808823 T1830 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 808823 T1826 C341 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@2887623d main
[junit4:junit4] 2> 808824 T1826 C341 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808825 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2887623d main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 808825 T1826 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 808826 T1781 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 26
[junit4:junit4] 2> 808827 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808829 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808833 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808837 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808838 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808841 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808844 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808845 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808847 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808853 T1747 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325043&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808853 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325043&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808856 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325043&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808857 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808862 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325052&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808862 T1749 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325052&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808865 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325052&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[j
[...truncated too long message...]
it4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] 2> at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] 2> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=bg, timezone=SystemV/PST8PDT
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=132777480,total=456130560
[junit4:junit4] 2> NOTE: All tests run in this JVM: [AnalysisAfterCoreReloadTest, SuggesterTest, DocumentBuilderTest, BlockCacheTest, TestDocumentBuilder, TermVectorComponentTest, QueryParsingTest, SystemInfoHandlerTest, FileBasedSpellCheckerTest, BinaryUpdateRequestHandlerTest, RecoveryZkTest, MoreLikeThisHandlerTest, PluginInfoTest, ShardRoutingTest, SimpleFacetsTest, TestPluginEnable, BasicFunctionalityTest, TestFastWriter, TestSweetSpotSimilarityFactory, TestMaxScoreQueryParser, TestCopyFieldCollectionResource, DirectSolrConnectionTest, TestDynamicFieldCollectionResource, TestSolrQueryParser, TestFunctionQuery, WordBreakSolrSpellCheckerTest, ResponseLogComponentTest, CoreAdminHandlerTest, UpdateRequestProcessorFactoryTest, ZkCLITest, TestReplicationHandler, PolyFieldTest, TestSearchPerf, SyncSliceTest, CircularListTest, TestFuzzyAnalyzedSuggestions, UnloadDistributedZkTest, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=DC0881C3A2CB615A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=bg -Dtests.timezone=SystemV/PST8PDT -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([DC0881C3A2CB615A]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([DC0881C3A2CB615A]:0)
[junit4:junit4] Completed on J0 in 242.04s, 1 test, 2 errors <<< FAILURES!
[...truncated 618 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 811 ignored (4 assumptions)
Total time: 39 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4102 - Still Failing
Posted by Mark Miller <ma...@gmail.com>.
FreeBSD issue it looks - that just started recently - an http connect attempt that won't time out even though there is a timeout set. I need to try and dig into it on my FreeBSD machine when I get a chance.
- Mark
On Jul 1, 2013, at 9:16 AM, Apache Jenkins Server <je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4102/
>
> 2 tests failed.
> FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
>
> Error Message:
> 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
> 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
>
>
> FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
>
> Error Message:
> There are still zombie threads that couldn't be terminated: 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
> 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
>
>
>
>
> Build Log:
> [...truncated 9599 lines...]
> [junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
> [junit4:junit4] 2> 732978 T2067 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /zm/x
> [junit4:junit4] 2> 732981 T2067 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372684133825
> [junit4:junit4] 2> 732983 T2067 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4:junit4] 2> 732983 T2068 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4:junit4] 2> 733084 T2067 oasc.ZkTestServer.run start zk server on port:17602
> [junit4:junit4] 2> 733085 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 733090 T2074 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3329a45 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 733090 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 733091 T2067 oascc.SolrZkClient.makePath makePath: /solr
> [junit4:junit4] 2> 733108 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 733110 T2076 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4792a0c9 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 733110 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 733111 T2067 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4:junit4] 2> 733115 T2067 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4:junit4] 2> 733118 T2067 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4:junit4] 2> 733121 T2067 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4:junit4] 2> 733125 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 733125 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 733131 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
> [junit4:junit4] 2> 733132 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4:junit4] 2> 733240 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 733240 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 733245 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4:junit4] 2> 733246 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4:junit4] 2> 733249 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4:junit4] 2> 733250 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4:junit4] 2> 733257 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 733258 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 733261 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 733262 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 733265 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 733265 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 733273 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 733273 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 733458 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 733461 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:19896
> [junit4:junit4] 2> 733462 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 733462 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 733462 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121
> [junit4:junit4] 2> 733463 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/solr.xml
> [junit4:junit4] 2> 733463 T2067 oasc.CoreContainer.<init> New CoreContainer 1845363899
> [junit4:junit4] 2> 733463 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/'
> [junit4:junit4] 2> 733464 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/'
> [junit4:junit4] 2> 733550 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 733550 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 733553 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 733553 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 733564 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 733565 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 733565 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 733566 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 733569 T2087 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5766e024 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 733569 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 733584 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 733597 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 733599 T2089 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6974bbb6 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 733599 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 733602 T2067 oascc.SolrZkClient.makePath makePath: /overseer/queue
> [junit4:junit4] 2> 733611 T2067 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
> [junit4:junit4] 2> 733614 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4:junit4] 2> 733621 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19896_zm%2Fx
> [junit4:junit4] 2> 733623 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19896_zm%2Fx
> [junit4:junit4] 2> 733627 T2067 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4:junit4] 2> 733642 T2067 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 733646 T2067 oasc.Overseer.start Overseer (id=89960227394879491-127.0.0.1:19896_zm%2Fx-n_0000000000) starting
> [junit4:junit4] 2> 733656 T2067 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
> [junit4:junit4] 2> 733667 T2091 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 733667 T2067 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4:junit4] 2> 733670 T2067 oascc.SolrZkClient.makePath makePath: /aliases.json
> [junit4:junit4] 2> 733677 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 733680 T2090 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 733684 T2092 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 733684 T2092 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 733685 T2092 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 735184 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 735184 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:19896_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:19896/zm/x"}
> [junit4:junit4] 2> 735185 T2090 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
> [junit4:junit4] 2> 735185 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 735201 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 735688 T2092 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 735688 T2092 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1
> [junit4:junit4] 2> 735689 T2092 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4:junit4] 2> 735690 T2092 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 735690 T2092 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
> [junit4:junit4] 2> 735692 T2092 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/'
> [junit4:junit4] 2> 735693 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/lib/README' to classloader
> [junit4:junit4] 2> 735694 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 735746 T2092 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 735808 T2092 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 735911 T2092 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 735918 T2092 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 736551 T2092 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 736564 T2092 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 736568 T2092 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 736585 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 736590 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 736595 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 736597 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 736598 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 736598 T2092 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 736598 T2092 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/
> [junit4:junit4] 2> 736599 T2092 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 736600 T2092 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data
> [junit4:junit4] 2> 736600 T2092 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index/
> [junit4:junit4] 2> 736600 T2092 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 736601 T2092 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index
> [junit4:junit4] 2> 736605 T2092 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 736606 T2092 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 736608 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 736609 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 736609 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 736610 T2092 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 736610 T2092 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 736611 T2092 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 736611 T2092 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 736612 T2092 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 736612 T2092 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 736628 T2092 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 736636 T2092 oass.SolrIndexSearcher.<init> Opening Searcher@782f5cc2 main
> [junit4:junit4] 2> 736637 T2092 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 736637 T2092 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 736641 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@782f5cc2 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 736642 T2092 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 736643 T2092 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:19896/zm/x collection:control_collection shard:shard1
> [junit4:junit4] 2> 736643 T2092 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4:junit4] 2> 736662 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4:junit4] 2> 736665 T2092 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 736665 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 736665 T2092 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19896/zm/x/collection1/
> [junit4:junit4] 2> 736666 T2092 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 736666 T2092 oasc.SyncStrategy.syncToMe http://127.0.0.1:19896/zm/x/collection1/ has no replicas
> [junit4:junit4] 2> 736666 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19896/zm/x/collection1/ shard1
> [junit4:junit4] 2> 736667 T2092 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4:junit4] 2> 736707 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 736715 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 736724 T2092 oasc.ZkController.register We are http://127.0.0.1:19896/zm/x/collection1/ and leader is http://127.0.0.1:19896/zm/x/collection1/
> [junit4:junit4] 2> 736724 T2092 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19896/zm/x
> [junit4:junit4] 2> 736724 T2092 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 736724 T2092 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 736725 T2092 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 736726 T2092 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 736728 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 736728 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 736729 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 736746 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:
> [junit4:junit4] 2> 736748 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 736750 T2096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78c4fabc name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 736750 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 736752 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 736755 T2067 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
> [junit4:junit4] 2> 737034 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 737037 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16933
> [junit4:junit4] 2> 737038 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 737039 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 737039 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598
> [junit4:junit4] 2> 737040 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/solr.xml
> [junit4:junit4] 2> 737040 T2067 oasc.CoreContainer.<init> New CoreContainer 1071384450
> [junit4:junit4] 2> 737041 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/'
> [junit4:junit4] 2> 737041 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/'
> [junit4:junit4] 2> 737171 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 737172 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 737172 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 737173 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 737173 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 737174 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 737174 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 737175 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 737175 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 737176 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 737193 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 737194 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 737194 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 737195 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 737198 T2107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14fa8922 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 737199 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 737214 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 737227 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 737229 T2109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66491c84 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 737229 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 737241 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 738222 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 738223 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:19896_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:19896/zm/x"}
> [junit4:junit4] 2> 738245 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738245 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738245 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738247 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16933_zm%2Fx
> [junit4:junit4] 2> 738256 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16933_zm%2Fx
> [junit4:junit4] 2> 738260 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738260 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738260 T2096 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 738261 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 738262 T2109 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 738262 T2089 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 738269 T2110 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 738269 T2110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 738273 T2110 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 739760 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:16933_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:16933/zm/x"}
> [junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
> [junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 739781 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 739781 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 739781 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 740276 T2110 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 740276 T2110 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1
> [junit4:junit4] 2> 740276 T2110 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 740277 T2110 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 740277 T2110 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 740279 T2110 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/'
> [junit4:junit4] 2> 740280 T2110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/lib/README' to classloader
> [junit4:junit4] 2> 740281 T2110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 740333 T2110 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 740395 T2110 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 740497 T2110 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 740504 T2110 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 741168 T2110 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 741181 T2110 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 741185 T2110 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 741202 T2110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 741207 T2110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 741211 T2110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 741213 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 741213 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 741214 T2110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 741215 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 741215 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 741215 T2110 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 741216 T2110 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/
> [junit4:junit4] 2> 741216 T2110 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 741217 T2110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1
> [junit4:junit4] 2> 741217 T2110 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index/
> [junit4:junit4] 2> 741217 T2110 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 741218 T2110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index
> [junit4:junit4] 2> 741221 T2110 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 741221 T2110 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 741224 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 741224 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 741225 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 741226 T2110 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 741226 T2110 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 741227 T2110 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 741227 T2110 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 741228 T2110 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 741228 T2110 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 741243 T2110 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 741250 T2110 oass.SolrIndexSearcher.<init> Opening Searcher@68a89d83 main
> [junit4:junit4] 2> 741251 T2110 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 741251 T2110 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 741255 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68a89d83 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 741257 T2110 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 741257 T2110 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16933/zm/x collection:collection1 shard:shard1
> [junit4:junit4] 2> 741258 T2110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4:junit4] 2> 741278 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4:junit4] 2> 741281 T2110 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 741281 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 741281 T2110 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:16933/zm/x/collection1/
> [junit4:junit4] 2> 741282 T2110 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 741282 T2110 oasc.SyncStrategy.syncToMe http://127.0.0.1:16933/zm/x/collection1/ has no replicas
> [junit4:junit4] 2> 741282 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:16933/zm/x/collection1/ shard1
> [junit4:junit4] 2> 741282 T2110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4:junit4] 2> 741286 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 741295 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 741295 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 741295 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 741341 T2110 oasc.ZkController.register We are http://127.0.0.1:16933/zm/x/collection1/ and leader is http://127.0.0.1:16933/zm/x/collection1/
> [junit4:junit4] 2> 741341 T2110 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16933/zm/x
> [junit4:junit4] 2> 741341 T2110 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 741341 T2110 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 741342 T2110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 741344 T2110 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 741345 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 741346 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 741346 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 741609 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 741612 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45733
> [junit4:junit4] 2> 741613 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 741613 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 741614 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204
> [junit4:junit4] 2> 741614 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/solr.xml
> [junit4:junit4] 2> 741614 T2067 oasc.CoreContainer.<init> New CoreContainer 2090646828
> [junit4:junit4] 2> 741615 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/'
> [junit4:junit4] 2> 741615 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/'
> [junit4:junit4] 2> 741736 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 741737 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 741738 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 741738 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 741740 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 741740 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 741741 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 741755 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 741756 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 741756 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 741758 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 741761 T2123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2dd83276 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 741761 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 741779 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 741789 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 741792 T2125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c940331 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 741792 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 741798 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 742801 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45733_zm%2Fx
> [junit4:junit4] 2> 742802 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 742821 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45733_zm%2Fx
> [junit4:junit4] 2> 742821 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:16933_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:16933/zm/x"}
> [junit4:junit4] 2> 742863 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 742863 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 742863 T2125 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 742863 T2096 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 742864 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742882 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742882 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742922 T2109 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 742923 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742926 T2089 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 742927 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742928 T2126 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 742929 T2126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 742957 T2126 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 744428 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node2",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45733_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45733/zm/x"}
> [junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4] 2> 744450 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 744450 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 744450 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 744450 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 744959 T2126 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 744959 T2126 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1
> [junit4:junit4] 2> 744959 T2126 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 744960 T2126 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 744960 T2126 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 744962 T2126 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/'
> [junit4:junit4] 2> 744963 T2126 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/lib/README' to classloader
> [junit4:junit4] 2> 744964 T2126 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 745016 T2126 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 745078 T2126 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 745181 T2126 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 745188 T2126 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 745851 T2126 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 745863 T2126 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 745867 T2126 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 745883 T2126 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 745888 T2126 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 745892 T2126 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 745896 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 745896 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 745896 T2126 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 745896 T2126 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/
> [junit4:junit4] 2> 745897 T2126 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 745898 T2126 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2
> [junit4:junit4] 2> 745899 T2126 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index/
> [junit4:junit4] 2> 745899 T2126 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 745899 T2126 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index
> [junit4:junit4] 2> 745903 T2126 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@63f3a140 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a20d579),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 745903 T2126 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 745906 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 745906 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 745907 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 745907 T2126 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 745909 T2126 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 745909 T2126 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 745924 T2126 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 745931 T2126 oass.SolrIndexSearcher.<init> Opening Searcher@3d1a7e9d main
> [junit4:junit4] 2> 745932 T2126 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 745933 T2126 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 745937 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d1a7e9d main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 745939 T2126 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 745939 T2126 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45733/zm/x collection:collection1 shard:shard2
> [junit4:junit4] 2> 745940 T2126 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4:junit4] 2> 745958 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
> [junit4:junit4] 2> 745962 T2126 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 745962 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45733/zm/x/collection1/
> [junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.syncToMe http://127.0.0.1:45733/zm/x/collection1/ has no replicas
> [junit4:junit4] 2> 745963 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45733/zm/x/collection1/ shard2
> [junit4:junit4] 2> 745963 T2126 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4:junit4] 2> 747462 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 747480 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 747480 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 747480 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 747480 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 747506 T2126 oasc.ZkController.register We are http://127.0.0.1:45733/zm/x/collection1/ and leader is http://127.0.0.1:45733/zm/x/collection1/
> [junit4:junit4] 2> 747506 T2126 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45733/zm/x
> [junit4:junit4] 2> 747506 T2126 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 747506 T2126 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 747507 T2126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 747508 T2126 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 747510 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 747510 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 747511 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 747798 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 747801 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45777
> [junit4:junit4] 2> 747802 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 747802 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 747803 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371
> [junit4:junit4] 2> 747803 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/solr.xml
> [junit4:junit4] 2> 747804 T2067 oasc.CoreContainer.<init> New CoreContainer 805132591
> [junit4:junit4] 2> 747804 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/'
> [junit4:junit4] 2> 747805 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/'
> [junit4:junit4] 2> 747936 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 747937 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 747937 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 747938 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 747938 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 747939 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 747939 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 747940 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 747940 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 747941 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 747958 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 747959 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 747960 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 747961 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 747993 T2139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@616dff74 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 747993 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 748027 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 748037 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 748040 T2141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46e38d21 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 748040 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 748078 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 748987 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 748988 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node2",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45733_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45733/zm/x"}
> [junit4:junit4] 2> 749008 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749008 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749008 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749008 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749008 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749082 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45777_zm%2Fx
> [junit4:junit4] 2> 749084 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45777_zm%2Fx
> [junit4:junit4] 2> 749087 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749087 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749088 T2096 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749087 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749088 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 749088 T2125 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749089 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 749089 T2109 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749090 T2089 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749096 T2141 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749102 T2142 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 749102 T2142 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 749104 T2142 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 750514 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
> [junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 750533 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 750533 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 750533 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 750533 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 750533 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 751106 T2142 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 751106 T2142 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1
> [junit4:junit4] 2> 751106 T2142 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 751107 T2142 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 751107 T2142 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 751109 T2142 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/'
> [junit4:junit4] 2> 751110 T2142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 751111 T2142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/lib/README' to classloader
> [junit4:junit4] 2> 751163 T2142 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 751225 T2142 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 751328 T2142 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 751335 T2142 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 751970 T2142 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 751982 T2142 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 751987 T2142 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 752003 T2142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 752008 T2142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 752013 T2142 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 752014 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 752015 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 752015 T2142 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 752016 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 752016 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 752017 T2142 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 752017 T2142 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/
> [junit4:junit4] 2> 752017 T2142 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 752018 T2142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3
> [junit4:junit4] 2> 752019 T2142 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index/
> [junit4:junit4] 2> 752019 T2142 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 752020 T2142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index
> [junit4:junit4] 2> 752023 T2142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 752024 T2142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 752026 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 752027 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 752027 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 752028 T2142 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 752030 T2142 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 752030 T2142 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 752047 T2142 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 752054 T2142 oass.SolrIndexSearcher.<init> Opening Searcher@70a83d7a main
> [junit4:junit4] 2> 752055 T2142 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 752056 T2142 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 752060 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70a83d7a main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 752062 T2142 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 752062 T2142 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45777/zm/x collection:collection1 shard:shard1
> [junit4:junit4] 2> 752066 T2142 oasc.ZkController.register We are http://127.0.0.1:45777/zm/x/collection1/ and leader is http://127.0.0.1:16933/zm/x/collection1/
> [junit4:junit4] 2> 752066 T2142 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45777/zm/x
> [junit4:junit4] 2> 752066 T2142 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4] 2> 752067 T2142 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4] 2> ASYNC NEW_CORE C991 name=collection1 org.apache.solr.core.SolrCore@53b2b3a6 url=http://127.0.0.1:45777/zm/x/collection1 node=127.0.0.1:45777_zm%2Fx C991_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
> [junit4:junit4] 2> 752067 T2144 C991 P45777 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
> [junit4:junit4] 2> 752068 T2142 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 752068 T2144 C991 P45777 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4] 2> 752068 T2144 C991 P45777 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4] 2> 752069 T2144 C991 P45777 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 752069 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 752070 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 752070 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 752071 T2144 C991 P45777 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 752096 T2099 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
> [junit4:junit4] 2> 752360 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 752363 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:36101
> [junit4:junit4] 2> 752364 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 752365 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 752365 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938
> [junit4:junit4] 2> 752366 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/solr.xml
> [junit4:junit4] 2> 752366 T2067 oasc.CoreContainer.<init> New CoreContainer 1659676005
> [junit4:junit4] 2> 752367 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/'
> [junit4:junit4] 2> 752367 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/'
> [junit4:junit4] 2> 752496 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 752497 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 752498 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 752498 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 752499 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 752499 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 752500 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 752500 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 752501 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 752501 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 752518 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 752519 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 752520 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 752521 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 752524 T2156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cf00b8d name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 752525 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 752539 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 752552 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 752554 T2158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a15d897 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 752555 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 752566 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 753541 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 753542 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"recovering",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
> [junit4:junit4] 2> 753550 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753571 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36101_zm%2Fx
> [junit4:junit4] 2> 753573 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36101_zm%2Fx
> [junit4:junit4] 2> 753576 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753576 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753576 T2096 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753576 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753576 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753576 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 753576 T2125 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753577 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 753578 T2109 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753578 T2089 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753579 T2141 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753579 T2158 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753585 T2159 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 753585 T2159 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 753587 T2159 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 754099 T2099 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
> [junit4:junit4] 2> 754099 T2099 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:45777_zm%252Fx&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
> [junit4:junit4] 2> 755065 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
> [junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4] 2> 755072 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755590 T2159 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 755590 T2159 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1
> [junit4:junit4] 2> 755590 T2159 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 755591 T2159 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 755592 T2159 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 755593 T2159 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/'
> [junit4:junit4] 2> 755595 T2159 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/lib/README' to classloader
> [junit4:junit4] 2> 755595 T2159 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 755647 T2159 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 755709 T2159 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 755812 T2159 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 755819 T2159 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> ASYNC NEW_CORE C992 name=collection1 org.apache.solr.core.SolrCore@53b2b3a6 url=http://127.0.0.1:45777/zm/x/collection1 node=127.0.0.1:45777_zm%2Fx C992_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
> [junit4:junit4] 2> 756102 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:16933/zm/x/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4] 2> 756102 T2144 C992 P45777 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45777/zm/x START replicas=[http://127.0.0.1:16933/zm/x/collection1/] nUpdates=100
> [junit4:junit4] 2> 756103 T2144 C992 P45777 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
> [junit4:junit4] 2> 756103 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4] 2> 756103 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4] 2> 756104 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4] 2> 756104 T2144 C992 P45777 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4] 2> 756104 T2144 C992 P45777 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:16933/zm/x/collection1/. core=collection1
> [junit4:junit4] 2> 756105 T2144 C992 P45777 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> ASYNC NEW_CORE C993 name=collection1 org.apache.solr.core.SolrCore@215eb2d8 url=http://127.0.0.1:16933/zm/x/collection1 node=127.0.0.1:16933_zm%2Fx C993_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:16933_zm%2Fx, base_url=http://127.0.0.1:16933/zm/x, leader=true}
> [junit4:junit4] 2> 756113 T2100 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
> [junit4:junit4] 2> 756116 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 756119 T2101 C993 P16933 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 756120 T2101 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 756120 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 756121 T2101 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@4d16c784 realtime
> [junit4:junit4] 2> 756121 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 756122 T2101 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 756122 T2144 C992 P45777 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 756123 T2144 C992 P45777 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4] 2> 756125 T2102 C993 P16933 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 756125 T2102 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
> [junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 756128 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4] 2> 756453 T2159 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 756466 T2159 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 756470 T2159 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 756487 T2159 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 756492 T2159 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 756496 T2159 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 756498 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 756498 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 756499 T2159 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 756500 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 756500 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 756500 T2159 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 756501 T2159 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/
> [junit4:junit4] 2> 756501 T2159 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 756502 T2159 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4
> [junit4:junit4] 2> 756502 T2159 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index/
> [junit4:junit4] 2> 756503 T2159 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 756503 T2159 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index
> [junit4:junit4] 2> 756507 T2159 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3fb835d1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fdaa1ee),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 756507 T2159 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 756510 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 756510 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 756511 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 756513 T2159 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 756514 T2159 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 756514 T2159 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 756531 T2159 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 756538 T2159 oass.SolrIndexSearcher.<init> Opening Searcher@37878dd4 main
> [junit4:junit4] 2> 756539 T2159 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 756539 T2159 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 756544 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@37878dd4 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 756546 T2159 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 756546 T2159 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36101/zm/x collection:collection1 shard:shard2
> [junit4:junit4] 2> 756550 T2159 oasc.ZkController.register We are http://127.0.0.1:36101/zm/x/collection1/ and leader is http://127.0.0.1:45733/zm/x/collection1/
> [junit4:junit4] 2> 756550 T2159 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36101/zm/x
> [junit4:junit4] 2> 756550 T2159 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4] 2> 756550 T2159 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4] 2> ASYNC NEW_CORE C994 name=collection1 org.apache.solr.core.SolrCore@7f9607f7 url=http://127.0.0.1:36101/zm/x/collection1 node=127.0.0.1:36101_zm%2Fx C994_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
> [junit4:junit4] 2> 756551 T2162 C994 P36101 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
> [junit4:junit4] 2> 756551 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4] 2> 756551 T2159 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 756552 T2162 C994 P36101 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4] 2> 756552 T2162 C994 P36101 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 756553 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 756553 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 756553 T2162 C994 P36101 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 756554 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 756572 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 756573 T2115 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
> [junit4:junit4] 2> 756574 T2067 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
> [junit4:junit4] 2> 756575 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 756585 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 756585 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
> [junit4:junit4] 2> 756590 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"recovering",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
> [junit4:junit4] 2> 756601 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756602 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756602 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756602 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756601 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756601 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 757575 T2115 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
> [junit4:junit4] 2> 757575 T2115 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:36101_zm%252Fx&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
> [junit4:junit4] 2> 757578 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 758581 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> C994_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
> [junit4:junit4] 2> 759578 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:45733/zm/x/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4] 2> 759578 T2162 C994 P36101 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36101/zm/x START replicas=[http://127.0.0.1:45733/zm/x/collection1/] nUpdates=100
> [junit4:junit4] 2> 759579 T2162 C994 P36101 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
> [junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4] 2> 759580 T2162 C994 P36101 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4] 2> 759580 T2162 C994 P36101 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:45733/zm/x/collection1/. core=collection1
> [junit4:junit4] 2> 759581 T2162 C994 P36101 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 759584 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> ASYNC NEW_CORE C995 name=collection1 org.apache.solr.core.SolrCore@1e8d62e0 url=http://127.0.0.1:45733/zm/x/collection1 node=127.0.0.1:45733_zm%2Fx C995_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:45733_zm%2Fx, base_url=http://127.0.0.1:45733/zm/x, leader=true}
> [junit4:junit4] 2> 759588 T2116 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
> [junit4:junit4] 2> 759592 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 759594 T2117 C995 P45733 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@63f3a140 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a20d579),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 759596 T2117 C995 P45733 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 759596 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 759597 T2117 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@496ac93a realtime
> [junit4:junit4] 2> 759597 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 759597 T2117 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
> [junit4:junit4] 2> 759598 T2162 C994 P36101 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 759598 T2162 C994 P36101 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4] 2> 759600 T2118 C995 P45733 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 759600 T2118 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
> [junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 759602 T2162 C994 P36101 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 759639 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4] 2> 759641 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 759642 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
> [junit4:junit4] 2> 759653 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 760586 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760587 T2067 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
> [junit4:junit4] 2> ASYNC NEW_CORE C996 name=collection1 org.apache.solr.core.SolrCore@565d49dc url=http://127.0.0.1:19896/zm/x/collection1 node=127.0.0.1:19896_zm%2Fx C996_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:19896_zm%2Fx, base_url=http://127.0.0.1:19896/zm/x, leader=true}
> [junit4:junit4] 2> 760599 T2079 C996 P19896 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 760600 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 760602 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@462edf25 realtime
> [junit4:junit4] 2> 760602 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439363667266109440)} 0 6
> [junit4:junit4] 2> C994_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
> [junit4:junit4] 2> 760626 T2148 C994 P36101 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3fb835d1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fdaa1ee),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 760627 T2148 C994 P36101 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 760629 T2148 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@2c01e8e2 realtime
> [junit4:junit4] 2> 760629 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=FROMLEADER&_version_=-1439363667282886656&update.from=http://127.0.0.1:45733/zm/x/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667282886656)} 0 4
> [junit4:junit4] 2> C992_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
> [junit4:junit4] 2> 760630 T2131 C992 P45777 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 760631 T2131 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 760632 T2131 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@6925ec4e realtime
> [junit4:junit4] 2> 760633 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=FROMLEADER&_version_=-1439363667287080960&update.from=http://127.0.0.1:16933/zm/x/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667287080960)} 0 5
> [junit4:junit4] 2> 760634 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667287080960)} 0 17
> [junit4:junit4] 2> 760634 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439363667282886656)} 0 21
> [junit4:junit4] 2> 760636 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760638 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760640 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760642 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760643 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760646 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760648 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760652 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760674 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_2,generation=2}
> [junit4:junit4] 2> 760675 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4:junit4] 2> 760681 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@190c0090 main
> [junit4:junit4] 2> 760681 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760682 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@190c0090 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
> [junit4:junit4] 2> 760682 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667322732544)],commit=} 0 31
> [junit4:junit4] 2> 760689 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667358384128)]} 0 1
> [junit4:junit4] 2> 760690 T2104 C993 P16933 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 760692 T2105 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760692 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760692 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760693 T2131 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760693 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760693 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760695 T2150 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@693e8ddf main
> [junit4:junit4] 2> 760696 T2105 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_2,generation=2}
> [junit4:junit4] 2> 760696 T2121 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@10432ba8 main
> [junit4:junit4] 2> 760697 T2131 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_2,generation=2}
> [junit4:junit4] 2> 760697 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@693e8ddf main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760696 T2105 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4:junit4] 2> 760696 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760698 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10432ba8 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760697 T2131 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4:junit4] 2> 760697 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760699 T2105 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@6a162992 main
> [junit4:junit4] 2> 760698 T2150 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 760700 T2105 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760700 T2131 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@64c26d35 main
> [junit4:junit4] 2> 760700 T2121 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
> [junit4:junit4] 2> 760701 T2131 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760701 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a162992 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
> [junit4:junit4] 2> 760702 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64c26d35 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
> [junit4:junit4] 2> 760702 T2105 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 760703 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 760703 T2104 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667358384128)],commit=} 0 18
> [junit4:junit4] 2> 760705 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760707 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760712 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760715 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760717 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760720 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760723 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760724 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760726 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760732 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760732 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760736 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760737 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760743 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760743 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
> [junit4:junit4] 2> 760746 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760747 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760749 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760755 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760755 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760759 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760760 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760766 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760766 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760769 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760770 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760775 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760778 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_2,generation=2}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_3,generation=3}
> [junit4:junit4] 2> 760779 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
> [junit4:junit4] 2> 760780 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@54bd13e5 main
> [junit4:junit4] 2> 760781 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760781 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54bd13e5 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
> [junit4:junit4] 2> 760782 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667450658816)],commit=} 0 9
> [junit4:junit4] 2> 760795 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667467436032)]} 0 2
> [junit4:junit4] 2> 760796 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667467436032)]} 0 8
> [junit4:junit4] 2> 760796 T2120 C995 P45733 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 760798 T2133 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760798 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760800 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760800 T2185 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@17b4c540 main
> [junit4:junit4] 2> 760800 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760800 T2103 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760802 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17b4c540 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760802 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760801 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760803 T2148 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@67e514d main
> [junit4:junit4] 2> 760803 T2133 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_2,generation=2}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_3,generation=3}
> [junit4:junit4] 2> 760804 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760804 T2185 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 760805 T2103 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_2,generation=2}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_3,generation=3}
> [junit4:junit4] 2> 760805 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67e514d main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760805 T2133 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
> [junit4:junit4] 2> 760807 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
> [junit4:junit4] 2> 760806 T2103 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
> [junit4:junit4] 2> 760808 T2133 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@3cd97ef4 main
> [junit4:junit4] 2> 760809 T2133 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760809 T2103 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@1ec58f94 main
> [junit4:junit4] 2> 760810 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cd97ef4 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
> [junit4:junit4] 2> 760810 T2103 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760810 T2133 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
> [junit4:junit4] 2> 760811 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ec58f94 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
> [junit4:junit4] 2> 760811 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
> [junit4:junit4] 2> 760812 T2120 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 27
> [junit4:junit4] 2> 760813 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760815 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760819 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760823 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760824 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760828 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760832 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760833 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760835 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760841 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760841 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760846 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760847 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 760853 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760853 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760857 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760858 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760859 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760866 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760866 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760870 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760871 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760877 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760878 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760882 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760883 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 760889 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760892 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_3,generation=3}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_4,generation=4}
> [junit4:junit4] 2> 760893 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
> [junit4:junit4] 2> 760894 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@3b31fd2f main
> [junit4:junit4] 2> 760895 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760895 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b31fd2f main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
> [junit4:junit4] 2> 760896 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667570196480)],commit=} 0 9
> [junit4:junit4] 2> 760907 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667584876544)]} 0 2
> [junit4:junit4] 2> 760908 T2187 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45777/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667584876544)]} 0 7
> [junit4:junit4] 2> 760908 T2132 C992 P45777 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 760911 T2134 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760911 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760911 T2187 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760911 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760912 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760913 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760913 T2191 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@5dcf10fa main
> [junit4:junit4] 2> 760914 T2134 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_3,generation=3}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_4,generation=4}
> [junit4:junit4] 2> 760914 T2151 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@70a07912 main
> [junit4:junit4] 2> 760915 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5dcf10fa main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760915 T2187 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_3,generation=3}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_4,generation=4}
> [junit4:junit4] 2> 760916 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70a07912 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760914 T2134 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
> [junit4:junit4] 2> 760914 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760916 T2187 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
> [junit4:junit4] 2> 760915 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760917 T2134 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@6b4a3fc7 main
> [junit4:junit4] 2> 760917 T2191 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 760918 T2134 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760918 T2187 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@5338fea9 main
> [junit4:junit4] 2> 760918 T2151 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
> [junit4:junit4] 2> 760919 T2187 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760919 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b4a3fc7 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
> [junit4:junit4] 2> 760920 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5338fea9 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
> [junit4:junit4] 2> 760921 T2134 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 760921 T2187 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 760922 T2132 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 24
> [junit4:junit4] 2> 760923 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760925 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760929 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760931 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760933 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760936 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760939 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760940 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760942 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760948 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760949 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760952 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760953 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760958 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760959 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760962 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760963 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760964 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760970 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760972 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760975 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760976 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760981 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760982 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760985 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760986 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760991 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760994 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_4,generation=4}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_5,generation=5}
> [junit4:junit4] 2> 760994 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
> [junit4:junit4] 2> 760996 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@5657776 main
> [junit4:junit4] 2> 760996 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760997 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5657776 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
> [junit4:junit4] 2> 760998 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667677151232)],commit=} 0 9
> [junit4:junit4] 2> 761009 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667691831296)]} 0 2
> [junit4:junit4] 2> 761010 T2194 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:36101/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667691831296)]} 0 7
> [junit4:junit4] 2> 761010 T2149 C994 P36101 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 761013 T2198 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 761013 T2194 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 761013 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 761013 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 761014 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 761014 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 761015 T2152 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@4ba28ccb main
> [junit4:junit4] 2> 761015 T2199 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@2d76073e main
> [junit4:junit4] 2> 761016 T2194 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_4,generation=4}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_5,generation=5}
> [junit4:junit4] 2> 761017 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ba28ccb main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 761017 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d76073e main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 761016 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 761016 T2198 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_4,generation=4}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_5,generation=5}
> [junit4:junit4] 2> 761016 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 761018 T2198 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
> [junit4:junit4] 2> 761018 T2199 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
> [junit4:junit4] 2> 761017 T2194 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
> [junit4:junit4] 2> 761018 T2152 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
> [junit4:junit4] 2> 761020 T2198 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@23594073 main
> [junit4:junit4] 2> 761021 T2194 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@3b4fcbaa main
> [junit4:junit4] 2> 761021 T2198 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 761021 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23594073 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
> [junit4:junit4] 2> 761021 T2194 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 761022 T2198 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
> [junit4:junit4] 2> 761022 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b4fcbaa main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
> [junit4:junit4] 2> 761023 T2194 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 761024 T2149 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 24
> [junit4:junit4] 2> 761025 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761027 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761030 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761033 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761035 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761038 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761041 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761042 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761044 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761049 T2153 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761050 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761053 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 761054 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 761060 T2154 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761061 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761064 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 761065 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 761066 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761073 T2200 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761075 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761079 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 761080 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 761085 T2201 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761086 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761089 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 761090 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 761098 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[1 (1439363667787251712)]} 0 4
> [junit4:junit4] 2> 761111 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667796688896)]} 0 3
> [junit4:junit4] 2> 761112 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667796688896)]} 0 9
> [junit4:junit4] 2> 761112 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[1]} 0 12
> [junit4:junit4] 2> 761116 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[2 (1439363667809271808)]} 0 1
> [junit4:junit4] 2> 761128 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1439363667813466112)]} 0 3
> [junit4:junit4] 2> 761128 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[2 (1439363667813466112)]} 0 9
> [junit4:junit4] 2> 761132 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[3 (1439363667826049024)]} 0 1
> [junit4:junit4] 2> 761140 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1439363667830243328)]} 0 1
> [junit4:junit4] 2> 761141 T2119 C995 P4573
>
> [...truncated too long message...]
>
> TestBloomFilteredLucene41Postings), id=PostingsFormat(name=Memory doPackFST= true), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=PostingsFormat(name=TestBloomFilteredLucene41Postings), oddField_s=PostingsFormat(name=Memory doPackFST= true), sequence_i=PostingsFormat(name=Memory doPackFST= true), name=PostingsFormat(name=Memory doPackFST= true), foo_i=PostingsFormat(name=SimpleText), regex_dup_B_s=PostingsFormat(name=TestBloomFilteredLucene41Postings), multiDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tf1=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_dt1=PostingsFormat(name=SimpleText), genre_s=PostingsFormat(name=Memory doPackFST= true), author_t=PostingsFormat(name=SimpleText), range_facet_l=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_ti1=PostingsFormat(name=Memory doPackFST= true), text=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=SimpleText), val_i=PostingsFormat(name=Memory doPackFST= true), SubjectTerms_mfacet=PostingsFormat(name=SimpleText), series_t=PostingsFormat(name=SimpleText), a_t=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tdt1=PostingsFormat(name=SimpleText), regex_dup_A_s=PostingsFormat(name=Asserting), price=PostingsFormat(name=Asserting), other_tl1=PostingsFormat(name=SimpleText), n_l1=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_si=PostingsFormat(name=SimpleText), inStock=PostingsFormat(name=Asserting)}, docValues:{timestamp=DocValuesFormat(name=CheapBastard)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=es_CO, timezone=America/Bahia
> [junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=180932536,total=536870912
> [junit4:junit4] 2> NOTE: All tests run in this JVM: [TestBadConfig, TestPseudoReturnFields, TestIBSimilarityFactory, TestReplicationHandler, DocValuesTest, FileBasedSpellCheckerTest, BasicHdfsTest, PrimUtilsTest, TestPHPSerializedResponseWriter, TestLRUCache, SOLR749Test, ParsingFieldUpdateProcessorsTest, TestAddFieldRealTimeGet, DistributedTermsComponentTest, TestMaxScoreQueryParser, LoggingHandlerTest, ResourceLoaderTest, TestStressVersions, MinimalSchemaTest, TestDefaultSearchFieldResource, RegexBoostProcessorTest, TestFunctionQuery, PolyFieldTest, TestSolrDeletionPolicy1, ShowFileRequestHandlerTest, TestCopyFieldCollectionResource, TestQueryUtils, TestLFUCache, PrimitiveFieldTypeTest, TestTrie, TestSearchPerf, SuggesterTest, QueryEqualityTest, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, BlockDirectoryTest, SolrIndexSplitterTest, SyncSliceTest, BadCopyFieldTest, DirectUpdateHandlerOptimizeTest, SynonymTokenizerTest, TestManagedSchemaFieldResource, SuggesterFSTTest, PeerSyncTest, TestSolrDeletionPolicy2, TestGroupingSearch, TestDynamicFieldCollectionResource, AutoCommitTest, TestRandomDVFaceting, ShardRoutingTest, OverseerCollectionProcessorTest, TestFieldResource, DistributedSpellCheckComponentTest, TestSolrIndexConfig, CSVRequestHandlerTest, FieldMutatingUpdateProcessorTest, SimpleFacetsTest, TestXIncludeConfig, TestJoin, TestStressRecovery, TestWriterPerf, BasicDistributedZkTest]
> [junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=61B7523138BE4F06 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_CO -Dtests.timezone=America/Bahia -Dtests.file.encoding=ISO-8859-1
> [junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
> [junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
> [junit4:junit4] > 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> [junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> [junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
> [junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> [junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> [junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> [junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4] > at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
> [junit4:junit4] > 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> [junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> [junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
> [junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> [junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> [junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> [junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4] > at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
> [junit4:junit4] Completed on J0 in 242.77s, 1 test, 2 errors <<< FAILURES!
>
> [...truncated 533 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 628 ignored (5 assumptions)
>
> Total time: 40 minutes 55 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4102 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4102/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
Build Log:
[...truncated 9599 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 732978 T2067 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /zm/x
[junit4:junit4] 2> 732981 T2067 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372684133825
[junit4:junit4] 2> 732983 T2067 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 732983 T2068 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 733084 T2067 oasc.ZkTestServer.run start zk server on port:17602
[junit4:junit4] 2> 733085 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733090 T2074 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3329a45 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733090 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733091 T2067 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 733108 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733110 T2076 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4792a0c9 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733110 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733111 T2067 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 733115 T2067 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 733118 T2067 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 733121 T2067 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 733125 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 733125 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 733131 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 733132 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 733240 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 733240 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 733245 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 733246 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 733249 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 733250 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 733257 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 733258 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 733261 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 733262 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 733265 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 733265 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 733273 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 733273 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 733458 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 733461 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:19896
[junit4:junit4] 2> 733462 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 733462 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 733462 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121
[junit4:junit4] 2> 733463 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/solr.xml
[junit4:junit4] 2> 733463 T2067 oasc.CoreContainer.<init> New CoreContainer 1845363899
[junit4:junit4] 2> 733463 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/'
[junit4:junit4] 2> 733464 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/'
[junit4:junit4] 2> 733550 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 733550 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 733553 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 733553 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 733564 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 733565 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 733565 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 733566 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733569 T2087 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5766e024 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733569 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733584 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 733597 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733599 T2089 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6974bbb6 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733599 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733602 T2067 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 733611 T2067 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 733614 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 733621 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19896_zm%2Fx
[junit4:junit4] 2> 733623 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19896_zm%2Fx
[junit4:junit4] 2> 733627 T2067 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 733642 T2067 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 733646 T2067 oasc.Overseer.start Overseer (id=89960227394879491-127.0.0.1:19896_zm%2Fx-n_0000000000) starting
[junit4:junit4] 2> 733656 T2067 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 733667 T2091 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 733667 T2067 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 733670 T2067 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 733677 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 733680 T2090 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 733684 T2092 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 733684 T2092 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 733685 T2092 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 735184 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 735184 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:19896_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:19896/zm/x"}
[junit4:junit4] 2> 735185 T2090 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 735185 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 735201 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 735688 T2092 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 735688 T2092 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1
[junit4:junit4] 2> 735689 T2092 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 735690 T2092 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 735690 T2092 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 735692 T2092 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/'
[junit4:junit4] 2> 735693 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/lib/README' to classloader
[junit4:junit4] 2> 735694 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 735746 T2092 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 735808 T2092 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 735911 T2092 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 735918 T2092 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 736551 T2092 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 736564 T2092 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 736568 T2092 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 736585 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 736590 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 736595 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 736597 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 736598 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 736598 T2092 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 736598 T2092 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/
[junit4:junit4] 2> 736599 T2092 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 736600 T2092 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data
[junit4:junit4] 2> 736600 T2092 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index/
[junit4:junit4] 2> 736600 T2092 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 736601 T2092 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index
[junit4:junit4] 2> 736605 T2092 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
[junit4:junit4] 2> 736606 T2092 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 736608 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 736609 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 736609 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 736610 T2092 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 736610 T2092 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 736611 T2092 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 736611 T2092 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 736612 T2092 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 736612 T2092 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 736628 T2092 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 736636 T2092 oass.SolrIndexSearcher.<init> Opening Searcher@782f5cc2 main
[junit4:junit4] 2> 736637 T2092 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 736637 T2092 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 736641 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@782f5cc2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 736642 T2092 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 736643 T2092 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:19896/zm/x collection:control_collection shard:shard1
[junit4:junit4] 2> 736643 T2092 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 736662 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 736665 T2092 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 736665 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 736665 T2092 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19896/zm/x/collection1/
[junit4:junit4] 2> 736666 T2092 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 736666 T2092 oasc.SyncStrategy.syncToMe http://127.0.0.1:19896/zm/x/collection1/ has no replicas
[junit4:junit4] 2> 736666 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19896/zm/x/collection1/ shard1
[junit4:junit4] 2> 736667 T2092 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 736707 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 736715 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 736724 T2092 oasc.ZkController.register We are http://127.0.0.1:19896/zm/x/collection1/ and leader is http://127.0.0.1:19896/zm/x/collection1/
[junit4:junit4] 2> 736724 T2092 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19896/zm/x
[junit4:junit4] 2> 736724 T2092 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 736724 T2092 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 736725 T2092 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 736726 T2092 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 736728 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 736728 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 736729 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 736746 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 736748 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 736750 T2096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78c4fabc name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 736750 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 736752 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 736755 T2067 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 737034 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 737037 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16933
[junit4:junit4] 2> 737038 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 737039 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 737039 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598
[junit4:junit4] 2> 737040 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/solr.xml
[junit4:junit4] 2> 737040 T2067 oasc.CoreContainer.<init> New CoreContainer 1071384450
[junit4:junit4] 2> 737041 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/'
[junit4:junit4] 2> 737041 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/'
[junit4:junit4] 2> 737171 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 737172 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 737172 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 737173 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 737173 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 737174 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 737174 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 737175 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 737175 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 737176 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 737193 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 737194 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 737194 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 737195 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 737198 T2107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14fa8922 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 737199 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 737214 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 737227 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 737229 T2109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66491c84 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 737229 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 737241 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 738222 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 738223 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:19896_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:19896/zm/x"}
[junit4:junit4] 2> 738245 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738245 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738245 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738247 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16933_zm%2Fx
[junit4:junit4] 2> 738256 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16933_zm%2Fx
[junit4:junit4] 2> 738260 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738260 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738260 T2096 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 738261 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 738262 T2109 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 738262 T2089 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 738269 T2110 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 738269 T2110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 738273 T2110 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 739760 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:16933_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:16933/zm/x"}
[junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 739781 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 739781 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 739781 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 740276 T2110 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 740276 T2110 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1
[junit4:junit4] 2> 740276 T2110 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 740277 T2110 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 740277 T2110 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 740279 T2110 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/'
[junit4:junit4] 2> 740280 T2110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/lib/README' to classloader
[junit4:junit4] 2> 740281 T2110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 740333 T2110 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 740395 T2110 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 740497 T2110 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 740504 T2110 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 741168 T2110 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 741181 T2110 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 741185 T2110 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 741202 T2110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 741207 T2110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 741211 T2110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 741213 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 741213 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 741214 T2110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 741215 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 741215 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 741215 T2110 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 741216 T2110 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/
[junit4:junit4] 2> 741216 T2110 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 741217 T2110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1
[junit4:junit4] 2> 741217 T2110 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index/
[junit4:junit4] 2> 741217 T2110 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 741218 T2110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index
[junit4:junit4] 2> 741221 T2110 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> 741221 T2110 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 741224 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 741224 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 741225 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 741226 T2110 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 741226 T2110 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 741227 T2110 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 741227 T2110 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 741228 T2110 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 741228 T2110 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 741243 T2110 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 741250 T2110 oass.SolrIndexSearcher.<init> Opening Searcher@68a89d83 main
[junit4:junit4] 2> 741251 T2110 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 741251 T2110 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 741255 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68a89d83 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 741257 T2110 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 741257 T2110 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16933/zm/x collection:collection1 shard:shard1
[junit4:junit4] 2> 741258 T2110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 741278 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 741281 T2110 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 741281 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 741281 T2110 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:16933/zm/x/collection1/
[junit4:junit4] 2> 741282 T2110 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 741282 T2110 oasc.SyncStrategy.syncToMe http://127.0.0.1:16933/zm/x/collection1/ has no replicas
[junit4:junit4] 2> 741282 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:16933/zm/x/collection1/ shard1
[junit4:junit4] 2> 741282 T2110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 741286 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 741295 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 741295 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 741295 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 741341 T2110 oasc.ZkController.register We are http://127.0.0.1:16933/zm/x/collection1/ and leader is http://127.0.0.1:16933/zm/x/collection1/
[junit4:junit4] 2> 741341 T2110 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16933/zm/x
[junit4:junit4] 2> 741341 T2110 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 741341 T2110 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 741342 T2110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 741344 T2110 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 741345 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 741346 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 741346 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 741609 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 741612 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45733
[junit4:junit4] 2> 741613 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 741613 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 741614 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204
[junit4:junit4] 2> 741614 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/solr.xml
[junit4:junit4] 2> 741614 T2067 oasc.CoreContainer.<init> New CoreContainer 2090646828
[junit4:junit4] 2> 741615 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/'
[junit4:junit4] 2> 741615 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/'
[junit4:junit4] 2> 741736 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 741737 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 741738 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 741738 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 741740 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 741740 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 741741 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 741755 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 741756 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 741756 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 741758 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 741761 T2123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2dd83276 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 741761 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 741779 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 741789 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 741792 T2125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c940331 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 741792 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 741798 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 742801 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45733_zm%2Fx
[junit4:junit4] 2> 742802 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 742821 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45733_zm%2Fx
[junit4:junit4] 2> 742821 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:16933_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:16933/zm/x"}
[junit4:junit4] 2> 742863 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 742863 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 742863 T2125 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 742863 T2096 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 742864 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742882 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742882 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742922 T2109 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 742923 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742926 T2089 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 742927 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742928 T2126 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 742929 T2126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 742957 T2126 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 744428 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45733_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45733/zm/x"}
[junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 744450 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 744450 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 744450 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 744450 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 744959 T2126 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 744959 T2126 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1
[junit4:junit4] 2> 744959 T2126 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 744960 T2126 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 744960 T2126 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 744962 T2126 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/'
[junit4:junit4] 2> 744963 T2126 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/lib/README' to classloader
[junit4:junit4] 2> 744964 T2126 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 745016 T2126 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 745078 T2126 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 745181 T2126 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 745188 T2126 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 745851 T2126 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 745863 T2126 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 745867 T2126 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 745883 T2126 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 745888 T2126 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 745892 T2126 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 745896 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 745896 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 745896 T2126 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 745896 T2126 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/
[junit4:junit4] 2> 745897 T2126 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 745898 T2126 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2
[junit4:junit4] 2> 745899 T2126 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index/
[junit4:junit4] 2> 745899 T2126 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 745899 T2126 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index
[junit4:junit4] 2> 745903 T2126 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@63f3a140 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a20d579),segFN=segments_1,generation=1}
[junit4:junit4] 2> 745903 T2126 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 745906 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 745906 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 745907 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 745907 T2126 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 745909 T2126 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 745909 T2126 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 745924 T2126 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 745931 T2126 oass.SolrIndexSearcher.<init> Opening Searcher@3d1a7e9d main
[junit4:junit4] 2> 745932 T2126 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 745933 T2126 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 745937 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d1a7e9d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 745939 T2126 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 745939 T2126 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45733/zm/x collection:collection1 shard:shard2
[junit4:junit4] 2> 745940 T2126 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 745958 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 745962 T2126 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 745962 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45733/zm/x/collection1/
[junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.syncToMe http://127.0.0.1:45733/zm/x/collection1/ has no replicas
[junit4:junit4] 2> 745963 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45733/zm/x/collection1/ shard2
[junit4:junit4] 2> 745963 T2126 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 747462 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 747480 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 747480 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 747480 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 747480 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 747506 T2126 oasc.ZkController.register We are http://127.0.0.1:45733/zm/x/collection1/ and leader is http://127.0.0.1:45733/zm/x/collection1/
[junit4:junit4] 2> 747506 T2126 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45733/zm/x
[junit4:junit4] 2> 747506 T2126 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 747506 T2126 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 747507 T2126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 747508 T2126 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 747510 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 747510 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 747511 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 747798 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 747801 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45777
[junit4:junit4] 2> 747802 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 747802 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 747803 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371
[junit4:junit4] 2> 747803 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/solr.xml
[junit4:junit4] 2> 747804 T2067 oasc.CoreContainer.<init> New CoreContainer 805132591
[junit4:junit4] 2> 747804 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/'
[junit4:junit4] 2> 747805 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/'
[junit4:junit4] 2> 747936 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 747937 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 747937 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 747938 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 747938 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 747939 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 747939 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 747940 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 747940 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 747941 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 747958 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 747959 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 747960 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 747961 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 747993 T2139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@616dff74 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 747993 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 748027 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 748037 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 748040 T2141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46e38d21 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 748040 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 748078 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 748987 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 748988 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45733_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45733/zm/x"}
[junit4:junit4] 2> 749008 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749008 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749008 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749008 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749008 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749082 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45777_zm%2Fx
[junit4:junit4] 2> 749084 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45777_zm%2Fx
[junit4:junit4] 2> 749087 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749087 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749088 T2096 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749087 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749088 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 749088 T2125 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749089 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 749089 T2109 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749090 T2089 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749096 T2141 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749102 T2142 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 749102 T2142 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 749104 T2142 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 750514 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
[junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 750533 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 750533 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 750533 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 750533 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 750533 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 751106 T2142 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 751106 T2142 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1
[junit4:junit4] 2> 751106 T2142 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 751107 T2142 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 751107 T2142 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 751109 T2142 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/'
[junit4:junit4] 2> 751110 T2142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 751111 T2142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/lib/README' to classloader
[junit4:junit4] 2> 751163 T2142 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 751225 T2142 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 751328 T2142 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 751335 T2142 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 751970 T2142 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 751982 T2142 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 751987 T2142 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 752003 T2142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 752008 T2142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 752013 T2142 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 752014 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 752015 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 752015 T2142 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 752016 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 752016 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 752017 T2142 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 752017 T2142 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/
[junit4:junit4] 2> 752017 T2142 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 752018 T2142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3
[junit4:junit4] 2> 752019 T2142 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index/
[junit4:junit4] 2> 752019 T2142 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 752020 T2142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index
[junit4:junit4] 2> 752023 T2142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
[junit4:junit4] 2> 752024 T2142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 752026 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 752027 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 752027 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 752028 T2142 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 752030 T2142 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 752030 T2142 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 752047 T2142 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 752054 T2142 oass.SolrIndexSearcher.<init> Opening Searcher@70a83d7a main
[junit4:junit4] 2> 752055 T2142 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 752056 T2142 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 752060 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70a83d7a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 752062 T2142 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 752062 T2142 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45777/zm/x collection:collection1 shard:shard1
[junit4:junit4] 2> 752066 T2142 oasc.ZkController.register We are http://127.0.0.1:45777/zm/x/collection1/ and leader is http://127.0.0.1:16933/zm/x/collection1/
[junit4:junit4] 2> 752066 T2142 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45777/zm/x
[junit4:junit4] 2> 752066 T2142 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 752067 T2142 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C991 name=collection1 org.apache.solr.core.SolrCore@53b2b3a6 url=http://127.0.0.1:45777/zm/x/collection1 node=127.0.0.1:45777_zm%2Fx C991_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
[junit4:junit4] 2> 752067 T2144 C991 P45777 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 752068 T2142 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 752068 T2144 C991 P45777 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 752068 T2144 C991 P45777 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 752069 T2144 C991 P45777 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 752069 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 752070 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 752070 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 752071 T2144 C991 P45777 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 752096 T2099 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 752360 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 752363 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:36101
[junit4:junit4] 2> 752364 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 752365 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 752365 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938
[junit4:junit4] 2> 752366 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/solr.xml
[junit4:junit4] 2> 752366 T2067 oasc.CoreContainer.<init> New CoreContainer 1659676005
[junit4:junit4] 2> 752367 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/'
[junit4:junit4] 2> 752367 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/'
[junit4:junit4] 2> 752496 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 752497 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 752498 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 752498 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 752499 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 752499 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 752500 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 752500 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 752501 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 752501 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 752518 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 752519 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 752520 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 752521 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 752524 T2156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cf00b8d name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 752525 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 752539 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 752552 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 752554 T2158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a15d897 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 752555 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 752566 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 753541 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 753542 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
[junit4:junit4] 2> 753550 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753571 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36101_zm%2Fx
[junit4:junit4] 2> 753573 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36101_zm%2Fx
[junit4:junit4] 2> 753576 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753576 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753576 T2096 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753576 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753576 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753576 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 753576 T2125 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753577 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 753578 T2109 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753578 T2089 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753579 T2141 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753579 T2158 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753585 T2159 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 753585 T2159 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 753587 T2159 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 754099 T2099 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 754099 T2099 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:45777_zm%252Fx&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 755065 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
[junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 755072 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755590 T2159 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 755590 T2159 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1
[junit4:junit4] 2> 755590 T2159 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 755591 T2159 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 755592 T2159 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 755593 T2159 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/'
[junit4:junit4] 2> 755595 T2159 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/lib/README' to classloader
[junit4:junit4] 2> 755595 T2159 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 755647 T2159 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 755709 T2159 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 755812 T2159 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 755819 T2159 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C992 name=collection1 org.apache.solr.core.SolrCore@53b2b3a6 url=http://127.0.0.1:45777/zm/x/collection1 node=127.0.0.1:45777_zm%2Fx C992_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
[junit4:junit4] 2> 756102 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:16933/zm/x/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 756102 T2144 C992 P45777 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45777/zm/x START replicas=[http://127.0.0.1:16933/zm/x/collection1/] nUpdates=100
[junit4:junit4] 2> 756103 T2144 C992 P45777 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 756103 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 756103 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 756104 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 756104 T2144 C992 P45777 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 756104 T2144 C992 P45777 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:16933/zm/x/collection1/. core=collection1
[junit4:junit4] 2> 756105 T2144 C992 P45777 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C993 name=collection1 org.apache.solr.core.SolrCore@215eb2d8 url=http://127.0.0.1:16933/zm/x/collection1 node=127.0.0.1:16933_zm%2Fx C993_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:16933_zm%2Fx, base_url=http://127.0.0.1:16933/zm/x, leader=true}
[junit4:junit4] 2> 756113 T2100 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 756116 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 756119 T2101 C993 P16933 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> 756120 T2101 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 756120 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 756121 T2101 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@4d16c784 realtime
[junit4:junit4] 2> 756121 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 756122 T2101 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 756122 T2144 C992 P45777 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 756123 T2144 C992 P45777 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 756125 T2102 C993 P16933 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 756125 T2102 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 756128 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 756453 T2159 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 756466 T2159 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 756470 T2159 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 756487 T2159 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 756492 T2159 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 756496 T2159 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 756498 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 756498 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 756499 T2159 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 756500 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 756500 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 756500 T2159 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 756501 T2159 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/
[junit4:junit4] 2> 756501 T2159 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 756502 T2159 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4
[junit4:junit4] 2> 756502 T2159 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index/
[junit4:junit4] 2> 756503 T2159 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 756503 T2159 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index
[junit4:junit4] 2> 756507 T2159 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3fb835d1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fdaa1ee),segFN=segments_1,generation=1}
[junit4:junit4] 2> 756507 T2159 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 756510 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 756510 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 756511 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 756513 T2159 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 756514 T2159 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 756514 T2159 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 756531 T2159 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 756538 T2159 oass.SolrIndexSearcher.<init> Opening Searcher@37878dd4 main
[junit4:junit4] 2> 756539 T2159 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 756539 T2159 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 756544 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@37878dd4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 756546 T2159 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 756546 T2159 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36101/zm/x collection:collection1 shard:shard2
[junit4:junit4] 2> 756550 T2159 oasc.ZkController.register We are http://127.0.0.1:36101/zm/x/collection1/ and leader is http://127.0.0.1:45733/zm/x/collection1/
[junit4:junit4] 2> 756550 T2159 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36101/zm/x
[junit4:junit4] 2> 756550 T2159 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 756550 T2159 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C994 name=collection1 org.apache.solr.core.SolrCore@7f9607f7 url=http://127.0.0.1:36101/zm/x/collection1 node=127.0.0.1:36101_zm%2Fx C994_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
[junit4:junit4] 2> 756551 T2162 C994 P36101 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 756551 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 756551 T2159 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 756552 T2162 C994 P36101 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 756552 T2162 C994 P36101 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 756553 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 756553 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 756553 T2162 C994 P36101 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 756554 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 756572 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 756573 T2115 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 756574 T2067 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 756575 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 756585 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 756585 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
[junit4:junit4] 2> 756590 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
[junit4:junit4] 2> 756601 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756602 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756602 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756602 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756601 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756601 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 757575 T2115 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 757575 T2115 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:36101_zm%252Fx&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 757578 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 758581 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C994_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
[junit4:junit4] 2> 759578 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:45733/zm/x/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 759578 T2162 C994 P36101 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36101/zm/x START replicas=[http://127.0.0.1:45733/zm/x/collection1/] nUpdates=100
[junit4:junit4] 2> 759579 T2162 C994 P36101 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 759580 T2162 C994 P36101 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 759580 T2162 C994 P36101 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:45733/zm/x/collection1/. core=collection1
[junit4:junit4] 2> 759581 T2162 C994 P36101 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 759584 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C995 name=collection1 org.apache.solr.core.SolrCore@1e8d62e0 url=http://127.0.0.1:45733/zm/x/collection1 node=127.0.0.1:45733_zm%2Fx C995_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:45733_zm%2Fx, base_url=http://127.0.0.1:45733/zm/x, leader=true}
[junit4:junit4] 2> 759588 T2116 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 759592 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 759594 T2117 C995 P45733 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@63f3a140 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a20d579),segFN=segments_1,generation=1}
[junit4:junit4] 2> 759596 T2117 C995 P45733 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 759596 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 759597 T2117 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@496ac93a realtime
[junit4:junit4] 2> 759597 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 759597 T2117 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 759598 T2162 C994 P36101 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 759598 T2162 C994 P36101 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 759600 T2118 C995 P45733 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 759600 T2118 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 759602 T2162 C994 P36101 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 759639 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 759641 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 759642 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
[junit4:junit4] 2> 759653 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 760586 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760587 T2067 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C996 name=collection1 org.apache.solr.core.SolrCore@565d49dc url=http://127.0.0.1:19896/zm/x/collection1 node=127.0.0.1:19896_zm%2Fx C996_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:19896_zm%2Fx, base_url=http://127.0.0.1:19896/zm/x, leader=true}
[junit4:junit4] 2> 760599 T2079 C996 P19896 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
[junit4:junit4] 2> 760600 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 760602 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@462edf25 realtime
[junit4:junit4] 2> 760602 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439363667266109440)} 0 6
[junit4:junit4] 2> C994_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
[junit4:junit4] 2> 760626 T2148 C994 P36101 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3fb835d1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fdaa1ee),segFN=segments_1,generation=1}
[junit4:junit4] 2> 760627 T2148 C994 P36101 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 760629 T2148 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@2c01e8e2 realtime
[junit4:junit4] 2> 760629 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=FROMLEADER&_version_=-1439363667282886656&update.from=http://127.0.0.1:45733/zm/x/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667282886656)} 0 4
[junit4:junit4] 2> C992_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
[junit4:junit4] 2> 760630 T2131 C992 P45777 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
[junit4:junit4] 2> 760631 T2131 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 760632 T2131 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@6925ec4e realtime
[junit4:junit4] 2> 760633 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=FROMLEADER&_version_=-1439363667287080960&update.from=http://127.0.0.1:16933/zm/x/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667287080960)} 0 5
[junit4:junit4] 2> 760634 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667287080960)} 0 17
[junit4:junit4] 2> 760634 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439363667282886656)} 0 21
[junit4:junit4] 2> 760636 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760638 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760640 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760642 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760643 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760646 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760648 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760652 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760674 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_2,generation=2}
[junit4:junit4] 2> 760675 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 760681 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@190c0090 main
[junit4:junit4] 2> 760681 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760682 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@190c0090 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 760682 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667322732544)],commit=} 0 31
[junit4:junit4] 2> 760689 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667358384128)]} 0 1
[junit4:junit4] 2> 760690 T2104 C993 P16933 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 760692 T2105 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760692 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760692 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760693 T2131 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760693 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760693 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760695 T2150 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@693e8ddf main
[junit4:junit4] 2> 760696 T2105 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_2,generation=2}
[junit4:junit4] 2> 760696 T2121 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@10432ba8 main
[junit4:junit4] 2> 760697 T2131 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_2,generation=2}
[junit4:junit4] 2> 760697 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@693e8ddf main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760696 T2105 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 760696 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760698 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10432ba8 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760697 T2131 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 760697 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760699 T2105 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@6a162992 main
[junit4:junit4] 2> 760698 T2150 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 760700 T2105 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760700 T2131 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@64c26d35 main
[junit4:junit4] 2> 760700 T2121 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 760701 T2131 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760701 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a162992 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 760702 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64c26d35 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 760702 T2105 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 760703 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 760703 T2104 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667358384128)],commit=} 0 18
[junit4:junit4] 2> 760705 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760707 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760712 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760715 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760717 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760720 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760723 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760724 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760726 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760732 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760732 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760736 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760737 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760743 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760743 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 760746 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760747 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760749 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760755 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760755 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760759 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760760 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760766 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760766 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760769 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760770 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760775 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760778 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_3,generation=3}
[junit4:junit4] 2> 760779 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 760780 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@54bd13e5 main
[junit4:junit4] 2> 760781 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760781 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54bd13e5 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 760782 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667450658816)],commit=} 0 9
[junit4:junit4] 2> 760795 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667467436032)]} 0 2
[junit4:junit4] 2> 760796 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667467436032)]} 0 8
[junit4:junit4] 2> 760796 T2120 C995 P45733 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 760798 T2133 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760798 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760800 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760800 T2185 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@17b4c540 main
[junit4:junit4] 2> 760800 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760800 T2103 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760802 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17b4c540 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760802 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760801 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760803 T2148 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@67e514d main
[junit4:junit4] 2> 760803 T2133 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_3,generation=3}
[junit4:junit4] 2> 760804 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760804 T2185 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 760805 T2103 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_3,generation=3}
[junit4:junit4] 2> 760805 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67e514d main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760805 T2133 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 760807 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 760806 T2103 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 760808 T2133 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@3cd97ef4 main
[junit4:junit4] 2> 760809 T2133 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760809 T2103 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@1ec58f94 main
[junit4:junit4] 2> 760810 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cd97ef4 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 760810 T2103 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760810 T2133 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 760811 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ec58f94 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 760811 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 760812 T2120 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 27
[junit4:junit4] 2> 760813 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760815 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760819 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760823 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760824 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760828 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760832 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760833 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760835 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760841 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760841 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760846 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760847 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 760853 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760853 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760857 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760858 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760859 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760866 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760866 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760870 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760871 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760877 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760878 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760882 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760883 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 760889 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760892 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_4,generation=4}
[junit4:junit4] 2> 760893 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 760894 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@3b31fd2f main
[junit4:junit4] 2> 760895 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760895 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b31fd2f main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 760896 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667570196480)],commit=} 0 9
[junit4:junit4] 2> 760907 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667584876544)]} 0 2
[junit4:junit4] 2> 760908 T2187 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45777/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667584876544)]} 0 7
[junit4:junit4] 2> 760908 T2132 C992 P45777 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 760911 T2134 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760911 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760911 T2187 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760911 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760912 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760913 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760913 T2191 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@5dcf10fa main
[junit4:junit4] 2> 760914 T2134 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_4,generation=4}
[junit4:junit4] 2> 760914 T2151 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@70a07912 main
[junit4:junit4] 2> 760915 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5dcf10fa main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760915 T2187 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_4,generation=4}
[junit4:junit4] 2> 760916 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70a07912 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760914 T2134 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 760914 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760916 T2187 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 760915 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760917 T2134 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@6b4a3fc7 main
[junit4:junit4] 2> 760917 T2191 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 760918 T2134 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760918 T2187 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@5338fea9 main
[junit4:junit4] 2> 760918 T2151 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 760919 T2187 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760919 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b4a3fc7 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 760920 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5338fea9 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 760921 T2134 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 760921 T2187 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 760922 T2132 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 24
[junit4:junit4] 2> 760923 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760925 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760929 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760931 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760933 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760936 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760939 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760940 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760942 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760948 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760949 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760952 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760953 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760958 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760959 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760962 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760963 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760964 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760970 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760972 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760975 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760976 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760981 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760982 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760985 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760986 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760991 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760994 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_5,generation=5}
[junit4:junit4] 2> 760994 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 760996 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@5657776 main
[junit4:junit4] 2> 760996 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760997 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5657776 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 760998 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667677151232)],commit=} 0 9
[junit4:junit4] 2> 761009 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667691831296)]} 0 2
[junit4:junit4] 2> 761010 T2194 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:36101/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667691831296)]} 0 7
[junit4:junit4] 2> 761010 T2149 C994 P36101 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 761013 T2198 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 761013 T2194 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 761013 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 761013 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 761014 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 761014 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 761015 T2152 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@4ba28ccb main
[junit4:junit4] 2> 761015 T2199 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@2d76073e main
[junit4:junit4] 2> 761016 T2194 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_5,generation=5}
[junit4:junit4] 2> 761017 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ba28ccb main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 761017 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d76073e main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 761016 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 761016 T2198 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_5,generation=5}
[junit4:junit4] 2> 761016 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 761018 T2198 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 761018 T2199 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 761017 T2194 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 761018 T2152 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 761020 T2198 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@23594073 main
[junit4:junit4] 2> 761021 T2194 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@3b4fcbaa main
[junit4:junit4] 2> 761021 T2198 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 761021 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23594073 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 761021 T2194 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 761022 T2198 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 761022 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b4fcbaa main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 761023 T2194 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 761024 T2149 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 24
[junit4:junit4] 2> 761025 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761027 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761030 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761033 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761035 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761038 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761041 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761042 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761044 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761049 T2153 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761050 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761053 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 761054 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 761060 T2154 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761061 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761064 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 761065 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 761066 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761073 T2200 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761075 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761079 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 761080 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 761085 T2201 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761086 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761089 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 761090 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 761098 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[1 (1439363667787251712)]} 0 4
[junit4:junit4] 2> 761111 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667796688896)]} 0 3
[junit4:junit4] 2> 761112 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667796688896)]} 0 9
[junit4:junit4] 2> 761112 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[1]} 0 12
[junit4:junit4] 2> 761116 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[2 (1439363667809271808)]} 0 1
[junit4:junit4] 2> 761128 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1439363667813466112)]} 0 3
[junit4:junit4] 2> 761128 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[2 (1439363667813466112)]} 0 9
[junit4:junit4] 2> 761132 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[3 (1439363667826049024)]} 0 1
[junit4:junit4] 2> 761140 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1439363667830243328)]} 0 1
[junit4:junit4] 2> 761141 T2119 C995 P4573
[...truncated too long message...]
TestBloomFilteredLucene41Postings), id=PostingsFormat(name=Memory doPackFST= true), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=PostingsFormat(name=TestBloomFilteredLucene41Postings), oddField_s=PostingsFormat(name=Memory doPackFST= true), sequence_i=PostingsFormat(name=Memory doPackFST= true), name=PostingsFormat(name=Memory doPackFST= true), foo_i=PostingsFormat(name=SimpleText), regex_dup_B_s=PostingsFormat(name=TestBloomFilteredLucene41Postings), multiDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tf1=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_dt1=PostingsFormat(name=SimpleText), genre_s=PostingsFormat(name=Memory doPackFST= true), author_t=PostingsFormat(name=SimpleText), range_facet_l=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_ti1=PostingsFormat(name=Memory doPackFST= true), text=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=SimpleText), val_i=PostingsFormat(name=Memory doPackFST= true), SubjectTerms_mfacet=PostingsFormat(name=SimpleText), series_t=PostingsFormat(name=SimpleText), a_t=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tdt1=PostingsFormat(name=SimpleText), regex_dup_A_s=PostingsFormat(name=Asserting), price=PostingsFormat(name=Asserting), other_tl1=PostingsFormat(name=SimpleText), n_l1=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_si=PostingsFormat(name=SimpleText), inStock=PostingsFormat(name=Asserting)}, docValues:{timestamp=DocValuesFormat(name=CheapBastard)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=es_CO, timezone=America/Bahia
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=180932536,total=536870912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestBadConfig, TestPseudoReturnFields, TestIBSimilarityFactory, TestReplicationHandler, DocValuesTest, FileBasedSpellCheckerTest, BasicHdfsTest, PrimUtilsTest, TestPHPSerializedResponseWriter, TestLRUCache, SOLR749Test, ParsingFieldUpdateProcessorsTest, TestAddFieldRealTimeGet, DistributedTermsComponentTest, TestMaxScoreQueryParser, LoggingHandlerTest, ResourceLoaderTest, TestStressVersions, MinimalSchemaTest, TestDefaultSearchFieldResource, RegexBoostProcessorTest, TestFunctionQuery, PolyFieldTest, TestSolrDeletionPolicy1, ShowFileRequestHandlerTest, TestCopyFieldCollectionResource, TestQueryUtils, TestLFUCache, PrimitiveFieldTypeTest, TestTrie, TestSearchPerf, SuggesterTest, QueryEqualityTest, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, BlockDirectoryTest, SolrIndexSplitterTest, SyncSliceTest, BadCopyFieldTest, DirectUpdateHandlerOptimizeTest, SynonymTokenizerTest, TestManagedSchemaFieldResource, SuggesterFSTTest, PeerSyncTest, TestSolrDeletionPolicy2, TestGroupingSearch, TestDynamicFieldCollectionResource, AutoCommitTest, TestRandomDVFaceting, ShardRoutingTest, OverseerCollectionProcessorTest, TestFieldResource, DistributedSpellCheckComponentTest, TestSolrIndexConfig, CSVRequestHandlerTest, FieldMutatingUpdateProcessorTest, SimpleFacetsTest, TestXIncludeConfig, TestJoin, TestStressRecovery, TestWriterPerf, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=61B7523138BE4F06 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_CO -Dtests.timezone=America/Bahia -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
[junit4:junit4] Completed on J0 in 242.77s, 1 test, 2 errors <<< FAILURES!
[...truncated 533 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 628 ignored (5 assumptions)
Total time: 40 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4101 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4101/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([9340903FD7C7D6D8]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([9340903FD7C7D6D8]:0)
Build Log:
[...truncated 9181 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 94969 T521 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 94975 T521 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372638152823
[junit4:junit4] 2> 94976 T521 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 94976 T522 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 95077 T521 oasc.ZkTestServer.run start zk server on port:24079
[junit4:junit4] 2> 95078 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 95083 T528 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f762a9e name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 95083 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 95084 T521 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 95101 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 95102 T530 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13d6e17a name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 95103 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 95103 T521 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 95113 T521 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 95117 T521 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 95119 T521 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 95128 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 95129 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 95142 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 95143 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 95247 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 95248 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 95251 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 95252 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 95262 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 95263 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 95265 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 95266 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 95270 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 95270 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 95273 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 95274 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 95277 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 95278 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 96196 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 96460 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:24082
[junit4:junit4] 2> 96504 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 96505 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 96505 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135
[junit4:junit4] 2> 96506 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/solr.xml
[junit4:junit4] 2> 96506 T521 oasc.CoreContainer.<init> New CoreContainer 318197175
[junit4:junit4] 2> 96507 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/'
[junit4:junit4] 2> 96507 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/'
[junit4:junit4] 2> 96679 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 96680 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 96680 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 96681 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 96681 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 96682 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 96682 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 96682 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 96683 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 96732 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 97215 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 97216 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 97230 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 97231 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 97235 T542 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1164ad0 name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 97235 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 97241 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 97255 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 97257 T544 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd4eeea name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 97258 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 97260 T521 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 97274 T521 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 97278 T521 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 97281 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:24082_
[junit4:junit4] 2> 97289 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:24082_
[junit4:junit4] 2> 97293 T521 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 97305 T521 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 97315 T521 oasc.Overseer.start Overseer (id=89957213983866883-127.0.0.1:24082_-n_0000000000) starting
[junit4:junit4] 2> 97327 T521 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 97330 T546 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 97331 T521 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 97334 T521 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 97340 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 97342 T545 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 97383 T547 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 97383 T547 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 97385 T547 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 98846 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 98847 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24082_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24082"}
[junit4:junit4] 2> 98847 T545 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 98847 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 98868 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 99387 T547 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 99387 T547 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1
[junit4:junit4] 2> 99388 T547 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 99389 T547 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 99389 T547 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 99392 T547 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1/'
[junit4:junit4] 2> 99394 T547 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 99395 T547 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1/lib/README' to classloader
[junit4:junit4] 2> 99517 T547 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 99604 T547 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 99706 T547 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 99769 T547 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 100565 T547 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 100587 T547 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 100592 T547 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 100625 T547 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 100632 T547 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 100639 T547 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 100640 T547 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 100640 T547 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 100641 T547 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 100642 T547 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 100642 T547 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 100643 T547 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 100643 T547 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data/
[junit4:junit4] 2> 100644 T547 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 100646 T547 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data
[junit4:junit4] 2> 100647 T547 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data/index/
[junit4:junit4] 2> 100647 T547 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 100648 T547 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data/index
[junit4:junit4] 2> 100654 T547 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_1,generation=1}
[junit4:junit4] 2> 100654 T547 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 100659 T547 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 100659 T547 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 100661 T547 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 100662 T547 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 100663 T547 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 100663 T547 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 100664 T547 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 100665 T547 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 100666 T547 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 100683 T547 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 100692 T547 oass.SolrIndexSearcher.<init> Opening Searcher@28031121 main
[junit4:junit4] 2> 100694 T547 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 100694 T547 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 100702 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@28031121 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 100707 T547 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 100707 T547 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:24082 collection:control_collection shard:shard1
[junit4:junit4] 2> 100719 T547 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 100745 T547 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 100765 T547 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 100768 T547 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 100768 T547 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 100769 T547 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:24082/collection1/
[junit4:junit4] 2> 100769 T547 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 100770 T547 oasc.SyncStrategy.syncToMe http://127.0.0.1:24082/collection1/ has no replicas
[junit4:junit4] 2> 100770 T547 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:24082/collection1/ shard1
[junit4:junit4] 2> 100771 T547 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 101874 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 101923 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 101958 T547 oasc.ZkController.register We are http://127.0.0.1:24082/collection1/ and leader is http://127.0.0.1:24082/collection1/
[junit4:junit4] 2> 101958 T547 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:24082
[junit4:junit4] 2> 101959 T547 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 101959 T547 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 101959 T547 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 101961 T547 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 101968 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 101969 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 102069 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 102098 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 102106 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 102113 T551 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34ed0665 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 102113 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 102114 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 102119 T521 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 102391 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 102394 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:24086
[junit4:junit4] 2> 102395 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 102396 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 102396 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967
[junit4:junit4] 2> 102397 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/solr.xml
[junit4:junit4] 2> 102398 T521 oasc.CoreContainer.<init> New CoreContainer 1681937453
[junit4:junit4] 2> 102398 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/'
[junit4:junit4] 2> 102399 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/'
[junit4:junit4] 2> 102550 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 102551 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 102552 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 102552 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 102553 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 102553 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 102554 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 102555 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 102555 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 102556 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 102576 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 102577 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 102577 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 102578 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 102581 T562 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f751b15 name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 102582 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 102599 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 102614 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 102616 T564 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14581b64 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 102617 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 102626 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 103434 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 103436 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24082_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24082"}
[junit4:junit4] 2> 103452 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103452 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103452 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103630 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:24086_
[junit4:junit4] 2> 103632 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:24086_
[junit4:junit4] 2> 103635 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103635 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103635 T544 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 103636 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 103636 T564 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 103644 T551 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 103651 T565 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 103651 T565 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 103653 T565 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 104956 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 104957 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24086_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24086"}
[junit4:junit4] 2> 104958 T545 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 104958 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 104977 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 104977 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 104977 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 105655 T565 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 105655 T565 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1
[junit4:junit4] 2> 105656 T565 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 105656 T565 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 105657 T565 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 105659 T565 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1/'
[junit4:junit4] 2> 105660 T565 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 105661 T565 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1/lib/README' to classloader
[junit4:junit4] 2> 105741 T565 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 105823 T565 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 105924 T565 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 105966 T565 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 106739 T565 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 106763 T565 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 106768 T565 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 106798 T565 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 106804 T565 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 106810 T565 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 106812 T565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 106812 T565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 106813 T565 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 106814 T565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 106814 T565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 106815 T565 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 106815 T565 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1/
[junit4:junit4] 2> 106816 T565 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 106818 T565 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1
[junit4:junit4] 2> 106818 T565 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1/index/
[junit4:junit4] 2> 106819 T565 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 106819 T565 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1/index
[junit4:junit4] 2> 106824 T565 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 106824 T565 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 106829 T565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 106830 T565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 106831 T565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 106833 T565 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 106834 T565 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 106834 T565 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 106835 T565 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 106836 T565 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 106837 T565 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 106854 T565 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 106863 T565 oass.SolrIndexSearcher.<init> Opening Searcher@6f23c5ef main
[junit4:junit4] 2> 106864 T565 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 106865 T565 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 106875 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f23c5ef main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 106881 T565 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 106881 T565 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:24086 collection:collection1 shard:shard1
[junit4:junit4] 2> 106882 T565 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 106935 T565 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 106938 T565 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 106938 T565 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 106939 T565 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:24086/collection1/
[junit4:junit4] 2> 106939 T565 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 106940 T565 oasc.SyncStrategy.syncToMe http://127.0.0.1:24086/collection1/ has no replicas
[junit4:junit4] 2> 106940 T565 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:24086/collection1/ shard1
[junit4:junit4] 2> 106940 T565 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 107983 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 108003 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 108003 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 108003 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 108017 T565 oasc.ZkController.register We are http://127.0.0.1:24086/collection1/ and leader is http://127.0.0.1:24086/collection1/
[junit4:junit4] 2> 108017 T565 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:24086
[junit4:junit4] 2> 108017 T565 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 108018 T565 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 108018 T565 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 108020 T565 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 108021 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 108022 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 108022 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 108319 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 108322 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:24090
[junit4:junit4] 2> 108323 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 108324 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 108325 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890
[junit4:junit4] 2> 108325 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/solr.xml
[junit4:junit4] 2> 108326 T521 oasc.CoreContainer.<init> New CoreContainer 450148949
[junit4:junit4] 2> 108327 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/'
[junit4:junit4] 2> 108327 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/'
[junit4:junit4] 2> 108490 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 108491 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 108491 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 108492 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 108492 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 108493 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 108493 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 108494 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 108495 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 108495 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 108515 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 108516 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 108516 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 108517 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 108521 T578 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c0e8d95 name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 108522 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 108536 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 108551 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 108553 T580 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c160110 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 108553 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 108564 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 109508 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 109509 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24086_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24086"}
[junit4:junit4] 2> 109525 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109525 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109525 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109525 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109568 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:24090_
[junit4:junit4] 2> 109570 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:24090_
[junit4:junit4] 2> 109573 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109573 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109573 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109573 T544 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 109574 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 109575 T564 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 109575 T551 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 109576 T580 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 109582 T581 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 109582 T581 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 109584 T581 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 111029 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 111030 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24090_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24090"}
[junit4:junit4] 2> 111031 T545 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 111031 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 111036 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 111036 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 111036 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 111036 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 111586 T581 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 111586 T581 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1
[junit4:junit4] 2> 111587 T581 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 111587 T581 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 111588 T581 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 111590 T581 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1/'
[junit4:junit4] 2> 111591 T581 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1/lib/README' to classloader
[junit4:junit4] 2> 111592 T581 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 111661 T581 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 111736 T581 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 111838 T581 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 111874 T581 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 112628 T581 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 112649 T581 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 112654 T581 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 112685 T581 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 112691 T581 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 112697 T581 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 112698 T581 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 112699 T581 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 112700 T581 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 112701 T581 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 112701 T581 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 112702 T581 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 112702 T581 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2/
[junit4:junit4] 2> 112702 T581 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 112704 T581 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2
[junit4:junit4] 2> 112704 T581 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2/index/
[junit4:junit4] 2> 112705 T581 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 112705 T581 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2/index
[junit4:junit4] 2> 112711 T581 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d217978 lockFactory=org.apache.lucene.store.NativeFSLockFactory@263a4b0c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 112711 T581 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 112716 T581 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 112717 T581 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 112718 T581 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 112719 T581 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 112720 T581 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 112721 T581 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 112721 T581 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 112723 T581 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 112724 T581 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 112740 T581 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 112748 T581 oass.SolrIndexSearcher.<init> Opening Searcher@652eaec main
[junit4:junit4] 2> 112750 T581 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 112750 T581 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 112761 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@652eaec main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 112767 T581 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 112767 T581 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:24090 collection:collection1 shard:shard2
[junit4:junit4] 2> 112768 T581 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 112790 T581 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 112801 T581 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 112801 T581 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 112802 T581 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:24090/collection1/
[junit4:junit4] 2> 112802 T581 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 112802 T581 oasc.SyncStrategy.syncToMe http://127.0.0.1:24090/collection1/ has no replicas
[junit4:junit4] 2> 112803 T581 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:24090/collection1/ shard2
[junit4:junit4] 2> 112803 T581 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 114052 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 114074 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 114074 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 114074 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 114074 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 114093 T581 oasc.ZkController.register We are http://127.0.0.1:24090/collection1/ and leader is http://127.0.0.1:24090/collection1/
[junit4:junit4] 2> 114093 T581 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:24090
[junit4:junit4] 2> 114093 T581 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 114094 T581 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 114094 T581 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 114096 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 114097 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 114098 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 114098 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 114388 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 114391 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21017
[junit4:junit4] 2> 114392 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 114393 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 114394 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965
[junit4:junit4] 2> 114394 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/solr.xml
[junit4:junit4] 2> 114395 T521 oasc.CoreContainer.<init> New CoreContainer 1496973014
[junit4:junit4] 2> 114396 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/'
[junit4:junit4] 2> 114396 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/'
[junit4:junit4] 2> 114543 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 114544 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 114544 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 114545 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 114545 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 114546 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 114546 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 114547 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 114548 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 114548 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 114568 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 114569 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 114569 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 114570 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 114573 T594 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c9532e1 name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 114574 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 114577 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 114591 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 114594 T596 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28c61629 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 114594 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 114606 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 115580 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 115581 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24090_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24090"}
[junit4:junit4] 2> 115602 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115602 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115602 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115602 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115602 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115610 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21017_
[junit4:junit4] 2> 115612 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21017_
[junit4:junit4] 2> 115614 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115615 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115615 T544 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115615 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115616 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 115616 T564 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115615 T596 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115617 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 115623 T580 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115624 T551 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115630 T597 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 115630 T597 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 115632 T597 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 117107 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 117108 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21017_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21017"}
[junit4:junit4] 2> 117109 T545 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 117109 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 117116 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117116 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117116 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117116 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117116 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117635 T597 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 117635 T597 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1
[junit4:junit4] 2> 117636 T597 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 117636 T597 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 117637 T597 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 117639 T597 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1/'
[junit4:junit4] 2> 117640 T597 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 117641 T597 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1/lib/README' to classloader
[junit4:junit4] 2> 117705 T597 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 117787 T597 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 117889 T597 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 117921 T597 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 118651 T597 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 118669 T597 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 118673 T597 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 118701 T597 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 118708 T597 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 118713 T597 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 118714 T597 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 118715 T597 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 118716 T597 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 118717 T597 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 118717 T597 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 118717 T597 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 118718 T597 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3/
[junit4:junit4] 2> 118718 T597 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 118720 T597 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3
[junit4:junit4] 2> 118720 T597 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3/index/
[junit4:junit4] 2> 118720 T597 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 118721 T597 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3/index
[junit4:junit4] 2> 118725 T597 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 118726 T597 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 118730 T597 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 118731 T597 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 118732 T597 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 118733 T597 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 118734 T597 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 118735 T597 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 118735 T597 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 118736 T597 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 118737 T597 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 118752 T597 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 118760 T597 oass.SolrIndexSearcher.<init> Opening Searcher@3a6e99c1 main
[junit4:junit4] 2> 118761 T597 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 118762 T597 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 118772 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3a6e99c1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 118778 T597 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 118778 T597 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21017 collection:collection1 shard:shard1
[junit4:junit4] 2> 118782 T597 oasc.ZkController.register We are http://127.0.0.1:21017/collection1/ and leader is http://127.0.0.1:24086/collection1/
[junit4:junit4] 2> 118782 T597 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21017
[junit4:junit4] 2> 118782 T597 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 118783 T597 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C2_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 118790 T599 C2 P21017 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 118790 T597 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 118791 T599 C2 P21017 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 118791 T599 C2 P21017 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 118792 T599 C2 P21017 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 118792 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 118793 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 118793 T599 C2 P21017 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 118793 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 119165 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 119169 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44764
[junit4:junit4] 2> 119170 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 119170 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 119171 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665
[junit4:junit4] 2> 119171 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/solr.xml
[junit4:junit4] 2> 119172 T521 oasc.CoreContainer.<init> New CoreContainer 341679870
[junit4:junit4] 2> 119173 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/'
[junit4:junit4] 2> 119173 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/'
[junit4:junit4] 2> 119363 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 119364 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 119364 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 119365 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 119365 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 119366 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 119366 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 119367 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 119367 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 119368 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 119391 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 119391 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 119392 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 119393 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 119396 T611 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46a2ca3e name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 119397 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 119414 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 119431 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 119435 T613 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62f5ae30 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 119435 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 119448 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 120124 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 120125 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21017_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21017"}
[junit4:junit4] 2> 120146 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120387 T554 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 120388 T554 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4:junit4] 2> 120389 T554 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:21017_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=8
[junit4:junit4] 2> 120451 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44764_
[junit4:junit4] 2> 120453 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44764_
[junit4:junit4] 2> 120455 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120456 T596 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120456 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 120455 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120455 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120455 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120456 T544 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120459 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 120465 T613 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120466 T580 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120466 T564 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120466 T551 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120480 T614 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 120480 T614 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 120482 T614 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 121650 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 121652 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44764_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44764"}
[junit4:junit4] 2> 121653 T545 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 121654 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 121689 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 122485 T614 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 122486 T614 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1
[junit4:junit4] 2> 122487 T614 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 122488 T614 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 122488 T614 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 122490 T614 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1/'
[junit4:junit4] 2> 122492 T614 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 122494 T614 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1/lib/README' to classloader
[junit4:junit4] 2> ASYNC NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C3_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 122568 T599 C3 P21017 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:24086/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 122582 T599 C3 P21017 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 122613 T599 C3 P21017 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:21017 START replicas=[http://127.0.0.1:24086/collection1/] nUpdates=100
[junit4:junit4] 2> 122613 T614 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 122629 T599 C3 P21017 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 122630 T599 C3 P21017 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 122630 T599 C3 P21017 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 122630 T599 C3 P21017 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 122631 T599 C3 P21017 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 122631 T599 C3 P21017 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:24086/collection1/. core=collection1
[junit4:junit4] 2> 122632 T599 C3 P21017 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@3c41d377 url=http://127.0.0.1:24086/collection1 node=127.0.0.1:24086_ C4_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24086_, base_url=http://127.0.0.1:24086, leader=true}
[junit4:junit4] 2> 122690 T555 C4 P24086 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=9
[junit4:junit4] 2> 122712 T614 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 122731 T556 C4 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 122736 T556 C4 P24086 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 122737 T556 C4 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 122737 T556 C4 P24086 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 122738 T556 C4 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@42a70f8c realtime
[junit4:junit4] 2> 122739 T556 C4 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 122739 T556 C4 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 122813 T614 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 122818 T599 C3 P21017 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 122818 T599 C3 P21017 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> 122848 T599 C3 P21017 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 122854 T614 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@3c41d377 url=http://127.0.0.1:24086/collection1 node=127.0.0.1:24086_ C5_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24086_, base_url=http://127.0.0.1:24086, leader=true}
[junit4:junit4] 2> 122945 T557 C5 P24086 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 122946 T557 C5 P24086 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=94
[junit4:junit4] 2> ASYNC NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C6_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 122948 T599 C6 P21017 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 122948 T599 C6 P21017 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 122949 T599 C6 P21017 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 122949 T599 C6 P21017 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 122957 T599 C6 P21017 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 123194 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 123196 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21017_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21017"}
[junit4:junit4] 2> 123223 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123702 T614 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 123721 T614 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 123726 T614 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 123757 T614 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 123763 T614 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 123769 T614 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 123771 T614 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 123771 T614 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 123772 T614 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 123773 T614 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 123773 T614 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 123774 T614 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 123774 T614 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4/
[junit4:junit4] 2> 123774 T614 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 123776 T614 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4
[junit4:junit4] 2> 123776 T614 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4/index/
[junit4:junit4] 2> 123777 T614 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 123777 T614 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4/index
[junit4:junit4] 2> 123782 T614 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@74510dba lockFactory=org.apache.lucene.store.NativeFSLockFactory@75c74de9),segFN=segments_1,generation=1}
[junit4:junit4] 2> 123782 T614 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 123787 T614 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 123788 T614 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 123789 T614 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 123791 T614 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 123792 T614 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 123793 T614 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 123800 T614 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 123802 T614 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 123802 T614 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 123819 T614 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 123828 T614 oass.SolrIndexSearcher.<init> Opening Searcher@e9fe846 main
[junit4:junit4] 2> 123829 T614 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 123830 T614 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 123842 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e9fe846 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 123847 T614 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 123848 T614 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44764 collection:collection1 shard:shard2
[junit4:junit4] 2> 123851 T614 oasc.ZkController.register We are http://127.0.0.1:44764/collection1/ and leader is http://127.0.0.1:24090/collection1/
[junit4:junit4] 2> 123852 T614 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44764
[junit4:junit4] 2> 123852 T614 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 123852 T614 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C7 name=collection1 org.apache.solr.core.SolrCore@182275f2 url=http://127.0.0.1:44764/collection1 node=127.0.0.1:44764_ C7_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:44764_, base_url=http://127.0.0.1:44764}
[junit4:junit4] 2> 123853 T617 C7 P44764 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 123854 T614 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 123854 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 123855 T617 C7 P44764 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 123855 T617 C7 P44764 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 123855 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 123856 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 123857 T617 C7 P44764 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 123857 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 123878 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 123881 T570 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 123881 T521 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 123883 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 124727 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 124728 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44764_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44764"}
[junit4:junit4] 2> 124749 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124750 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124750 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124749 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124749 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124749 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124883 T570 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 124883 T570 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:44764_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 124885 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 125887 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C7_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:44764_, base_url=http://127.0.0.1:44764}
[junit4:junit4] 2> 126885 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:24090/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 126885 T617 C7 P44764 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44764 START replicas=[http://127.0.0.1:24090/collection1/] nUpdates=100
[junit4:junit4] 2> 126887 T617 C7 P44764 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 126887 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 126887 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 126887 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 126888 T617 C7 P44764 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 126889 T617 C7 P44764 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:24090/collection1/. core=collection1
[junit4:junit4] 2> 126889 T617 C7 P44764 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 126890 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C8 name=collection1 org.apache.solr.core.SolrCore@6d20036b url=http://127.0.0.1:24090/collection1 node=127.0.0.1:24090_ C8_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24090_, base_url=http://127.0.0.1:24090, leader=true}
[junit4:junit4] 2> 126898 T571 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 126904 T572 C8 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 126908 T572 C8 P24090 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d217978 lockFactory=org.apache.lucene.store.NativeFSLockFactory@263a4b0c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 126909 T572 C8 P24090 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 126910 T572 C8 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 126910 T572 C8 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@3a5cd1d3 realtime
[junit4:junit4] 2> 126911 T572 C8 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 126911 T572 C8 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 126913 T617 C7 P44764 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 126913 T617 C7 P44764 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 126917 T573 C8 P24090 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 126918 T573 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 126919 T617 C7 P44764 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 126919 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 126920 T617 C7 P44764 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 126920 T617 C7 P44764 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 126922 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 127756 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 127757 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44764_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44764"}
[junit4:junit4] 2> 127777 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127892 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 127893 T521 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C9 name=collection1 org.apache.solr.core.SolrCore@5dd15fc4 url=http://127.0.0.1:24082/collection1 node=127.0.0.1:24082_ C9_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24082_, base_url=http://127.0.0.1:24082, leader=true}
[junit4:junit4] 2> 128081 T533 C9 P24082 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_1,generation=1}
[junit4:junit4] 2> 128082 T533 C9 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 128132 T533 C9 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@581f9a8c realtime
[junit4:junit4] 2> 128138 T533 C9 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439315458386821120)} 0 151
[junit4:junit4] 2> C7_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:44764_, base_url=http://127.0.0.1:44764}
[junit4:junit4] 2> 128210 T604 C7 P44764 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@74510dba lockFactory=org.apache.lucene.store.NativeFSLockFactory@75c74de9),segFN=segments_1,generation=1}
[junit4:junit4] 2> 128211 T604 C7 P44764 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 128213 T604 C7 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@3ddf06fd realtime
[junit4:junit4] 2> 128214 T604 C7 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439315458570321920&update.from=http://127.0.0.1:24090/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439315458570321920)} 0 8
[junit4:junit4] 2> 128215 T574 C8 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439315458570321920)} 0 18
[junit4:junit4] 2> ASYNC NEW_CORE C10 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C10_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 128218 T586 C10 P21017 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 128219 T586 C10 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 128221 T586 C10 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@22d9628f realtime
[junit4:junit4] 2> 128221 T586 C10 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439315458570321920&update.from=http://127.0.0.1:24086/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439315458570321920)} 0 6
[junit4:junit4] 2> ASYNC NEW_CORE C11 name=collection1 org.apache.solr.core.SolrCore@3c41d377 url=http://127.0.0.1:24086/collection1 node=127.0.0.1:24086_ C11_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24086_, base_url=http://127.0.0.1:24086, leader=true}
[junit4:junit4] 2> 128222 T558 C11 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439315458570321920)} 0 25
[junit4:junit4] 2> 128224 T603 C7 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 45
[junit4:junit4] 2> 128225 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128227 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128238 T559 C11 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=7
[junit4:junit4] 2> 128243 T587 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128244 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128248 T575 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 128252 T605 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128268 T533 C9 P24082 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128327 T533 C9 P24082 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_2,generation=2}
[junit4:junit4] 2> 128328 T533 C9 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 128471 T533 C9 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@1ecd4f54 main
[junit4:junit4] 2> 128473 T533 C9 P24082 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128475 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ecd4f54 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 128476 T533 C9 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315458639527936)],commit=} 0 219
[junit4:junit4] 2> 128506 T586 C10 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24086/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439315458867068928)]} 0 4
[junit4:junit4] 2> 128508 T559 C11 P24086 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24086/collection1/, StdNode: http://127.0.0.1:21017/collection1/, StdNode: http://127.0.0.1:24090/collection1/, StdNode: http://127.0.0.1:44764/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 128514 T560 C11 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128514 T576 C8 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128514 T606 C7 P44764 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128514 T586 C10 P21017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128517 T606 C7 P44764 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 128516 T576 C8 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 128518 T606 C7 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@44831670 main
[junit4:junit4] 2> 128519 T576 C8 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@579128ce main
[junit4:junit4] 2> 128520 T606 C7 P44764 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128522 T576 C8 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128522 T582 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:305)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 128523 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44831670 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 128525 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@579128ce main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 128525 T606 C7 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 128526 T576 C8 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 128529 T560 C11 P24086 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_2,generation=2}
[junit4:junit4] 2> 128529 T560 C11 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 128530 T586 C10 P21017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 128531 T586 C10 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 128533 T560 C11 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@2d62624e main
[junit4:junit4] 2> 128534 T560 C11 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128535 T586 C10 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@2e7c73e6 main
[junit4:junit4] 2> 128536 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d62624e main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 128536 T586 C10 P21017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128537 T560 C11 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> 128537 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2e7c73e6 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 128538 T586 C10 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 128540 T559 C11 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315458867068928)],commit=} 0 59
[junit4:junit4] 2> 128541 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128543 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128580 T559 C11 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=33
[junit4:junit4] 2> 128593 T587 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 128601 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128606 T575 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 128611 T605 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128612 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128614 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128701 T607 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186466&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=77
[junit4:junit4] 2> 128701 T588 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186466&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=77
[junit4:junit4] 2> 128740 T588 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186466&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 128741 T559 C11 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=122
[junit4:junit4] 2> 128752 T608 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186593&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128752 T589 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186593&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 128757 T589 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186593&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 128759 T587 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 128761 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128772 T609 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186613&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128772 T590 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186613&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 128777 T590 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186613&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 128779 T575 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 128789 T591 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186630&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 128789 T635 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186630&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128794 T591 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186630&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 128796 T605 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 128806 T533 C9 P24082 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128863 T533 C9 P24082 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_3,generation=3}
[junit4:junit4] 2> 128864 T533 C9 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 128867 T533 C9 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@c10417 main
[junit4:junit4] 2> ASYNC NEW_CORE C12 name=collection1 org.apache.solr.core.SolrCore@5dd15fc4 url=http://127.0.0.1:24082/collection1 node=127.0.0.1:24082_ C12_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24082_, base_url=http://127.0.0.1:24082, leader=true}
[junit4:junit4] 2> 128879 T533 C12 P24082 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128880 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c10417 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 128882 T533 C12 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315459202613248)],commit=} 0 81
[junit4:junit4] 2> ASYNC NEW_CORE C13 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C13_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 129050 T586 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24086/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439315459304325120)]} 0 144
[junit4:junit4] 2> ASYNC NEW_CORE C14 name=collection1 org.apache.solr.core.SolrCore@3c41d377 url=http://127.0.0.1:24086/collection1 node=127.0.0.1:24086_ C14_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24086_, base_url=http://127.0.0.1:24086, leader=true}
[junit4:junit4] 2> 129052 T554 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24090/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439315459304325120)]} 0 155
[junit4:junit4] 2> ASYNC NEW_CORE C15 name=collection1 org.apache.solr.core.SolrCore@6d20036b url=http://127.0.0.1:24090/collection1 node=127.0.0.1:24090_ C15_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24090_, base_url=http://127.0.0.1:24090, leader=true}
[junit4:junit4] 2> 129053 T575 C15 P24090 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24086/collection1/, StdNode: http://127.0.0.1:21017/collection1/, StdNode: http://127.0.0.1:24090/collection1/, StdNode: http://127.0.0.1:44764/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 129059 T592 C13 P21017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C16 name=collection1 org.apache.solr.core.SolrCore@182275f2 url=http://127.0.0.1:44764/collection1 node=127.0.0.1:44764_ C16_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:44764_, base_url=http://127.0.0.1:44764}
[junit4:junit4] 2> 129060 T604 C16 P44764 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129059 T554 C14 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129059 T570 C15 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129063 T604 C16 P44764 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129065 T570 C15 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129065 T604 C16 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@38be4062 main
[junit4:junit4] 2> 129066 T570 C15 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@d8b9fac main
[junit4:junit4] 2> 129066 T604 C16 P44764 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129067 T570 C15 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129067 T582 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:305)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 129068 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@38be4062 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129070 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d8b9fac main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129070 T604 C16 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 129071 T570 C15 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 129075 T592 C13 P21017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_3,generation=3}
[junit4:junit4] 2> 129075 T592 C13 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 129076 T554 C14 P24086 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_3,generation=3}
[junit4:junit4] 2> 129076 T554 C14 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 129079 T592 C13 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@2b41cdbc main
[junit4:junit4] 2> 129081 T592 C13 P21017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129081 T554 C14 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@766e5053 main
[junit4:junit4] 2> 129082 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2b41cdbc main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 129082 T554 C14 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129083 T592 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 129083 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@766e5053 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 129085 T554 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 129086 T575 C15 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 199
[junit4:junit4] 2> 129087 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129089 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129095 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129101 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129103 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129109 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129113 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129115 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129116 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129125 T572 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186968&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129126 T588 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186968&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129131 T588 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186968&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129133 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129141 T639 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186984&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129142 T589 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186984&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129147 T589 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186984&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129149 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129150 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129159 T640 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187002&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129160 T590 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187002&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129165 T590 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187002&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129167 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129175 T641 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187018&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129176 T591 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187018&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129181 T591 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187018&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129183 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129193 T533 C12 P24082 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129202 T533 C12 P24082 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_4,generation=4}
[junit4:junit4] 2> 129202 T533 C12 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 129206 T533 C12 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@41de3257 main
[junit4:junit4] 2> 129207 T533 C12 P24082 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129208 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@41de3257 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 129210 T533 C12 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315459608412160)],commit=} 0 22
[junit4:junit4] 2> 129232 T586 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24086/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439315459640918016)]} 0 5
[junit4:junit4] 2> 129234 T556 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:21017/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439315459640918016)]} 0 15
[junit4:junit4] 2> 129235 T587 C13 P21017 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24086/collection1/, StdNode: http://127.0.0.1:21017/collection1/, StdNode: http://127.0.0.1:24090/collection1/, StdNode: http://127.0.0.1:44764/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 129242 T556 C14 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129242 T647 C15 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129242 T648 C16 P44764 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129242 T646 C13 P21017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129244 T648 C16 P44764 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129243 T647 C15 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129246 T648 C16 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@1aaa4a39 main
[junit4:junit4] 2> 129246 T647 C15 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@3f3861fb main
[junit4:junit4] 2> 129247 T648 C16 P44764 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129248 T647 C15 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129249 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1aaa4a39 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129250 T648 C16 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 129251 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f3861fb main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129252 T556 C14 P24086 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_4,generation=4}
[junit4:junit4] 2> 129252 T647 C15 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 129253 T646 C13 P21017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_4,generation=4}
[junit4:junit4] 2> 129253 T556 C14 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 129254 T646 C13 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 129257 T556 C14 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@25fb025 main
[junit4:junit4] 2> 129258 T646 C13 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@531db190 main
[junit4:junit4] 2> 129258 T556 C14 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129260 T646 C13 P21017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129261 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25fb025 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 129261 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@531db190 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 129262 T556 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 129264 T646 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4] 2> 129265 T587 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 51
[junit4:junit4] 2> 129266 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129268 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129274 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129279 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129281 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129286 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129291 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129293 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129294 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129304 T607 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187146&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129304 T649 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187146&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129310 T649 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187146&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129312 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 129321 T650 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187163&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=2
[junit4:junit4] 2> 129321 T608 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187163&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129326 T650 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187163&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129328 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129330 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129339 T651 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187181&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129339 T609 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187181&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129345 T651 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187181&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 129346 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129356 T652 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187198&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129357 T635 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187198&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129362 T652 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187198&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129364 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 129374 T533 C12 P24082 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129381 T533 C12 P24082 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_5,generation=5}
[junit4:junit4] 2> 129382 T533 C12 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 129385 T533 C12 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@6063f086 main
[junit4:junit4] 2> 129386 T533 C12 P24082 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129388 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6063f086 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 129390 T533 C12 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315459798204416)],commit=} 0 21
[junit4:junit4] 2> 129411 T586 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24086/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439315459829661696)]} 0 5
[junit4:junit4] 2> 129413 T558 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44764/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439315459829661696)]} 0 15
[junit4:junit4] 2> 129414 T605 C16 P44764 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24086/collection1/, StdNode: http://127.0.0.1:21017/collection1/, StdNode: http://127.0.0.1:24090/collection1/, StdNode: http://127.0.0.1:44764/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 129418 T656 C16 P44764 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129418 T558 C14 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129419 T574 C15 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129418 T655 C13 P21017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129420 T574 C15 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129418 T656 C16 P44764 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129422 T574 C15 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@4b352dda main
[junit4:junit4] 2> 129423 T656 C16 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@62b1f62 main
[junit4:junit4] 2> 129423 T574 C15 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129424 T656 C16 P44764 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129426 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b352dda main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129426 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@62b1f62 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129427 T574 C15 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 129428 T558 C14 P24086 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_5,generation=5}
[junit4:junit4] 2> 129429 T558 C14 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 129427 T656 C16 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 129429 T655 C13 P21017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_5,generation=5}
[junit4:junit4] 2> 129430 T655 C13 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 129432 T558 C14 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@16364f main
[junit4:junit4] 2> 129433 T558 C14 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129433 T655 C13 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@cbcb654 main
[junit4:junit4] 2> 129435 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16364f main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 129435 T655 C13 P21017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129437 T558 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 19
[junit4:junit4] 2> 129437 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@cbcb654 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 129439 T655 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 21
[junit4:junit4] 2> 129440 T605 C16 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 46
[junit4:junit4] 2> 129441 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129443 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129449 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129454 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 129457 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129463 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129467 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129469 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129470 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129479 T607 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187321&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129480 T588 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187321&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129487 T588 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187321&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 129488 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 statu
[...truncated too long message...]
upt leaked threads:
[junit4:junit4] 2> 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> 231740 T520 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4] 2> 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] 2> at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] 2> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_EC, timezone=America/Cayman
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=229778352,total=337641472
[junit4:junit4] 2> NOTE: All tests run in this JVM: [OverseerCollectionProcessorTest, TestXIncludeConfig, TestMaxScoreQueryParser, OverseerTest, FastVectorHighlighterTest, UpdateRequestProcessorFactoryTest, TestSolrXml, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=9340903FD7C7D6D8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_EC -Dtests.timezone=America/Cayman -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([9340903FD7C7D6D8]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([9340903FD7C7D6D8]:0)
[junit4:junit4] Completed on J0 in 231.86s, 1 test, 2 errors <<< FAILURES!
[...truncated 905 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 930 ignored (1 assumption)
Total time: 43 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure