You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/07/23 10:38:08 UTC
[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 329 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/329/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=1665, name=recoveryCmdExecutor-792-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=1665, name=recoveryCmdExecutor-792-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([C4391FDD5D0E0708]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=1665, name=recoveryCmdExecutor-792-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=1665, name=recoveryCmdExecutor-792-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([C4391FDD5D0E0708]:0)
Build Log:
[...truncated 9600 lines...]
[junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4] 2> 586984 T1156 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /c_/uk
[junit4] 2> 586988 T1156 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1374567225438
[junit4] 2> 586989 T1156 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 586989 T1157 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 587090 T1156 oasc.ZkTestServer.run start zk server on port:30042
[junit4] 2> 587091 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 587098 T1163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a2048ea name:ZooKeeperConnection Watcher:127.0.0.1:30042 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 587098 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 587098 T1156 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 587118 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 587120 T1165 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5dd7480d name:ZooKeeperConnection Watcher:127.0.0.1:30042/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 587121 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 587121 T1156 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 587131 T1156 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 587134 T1156 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 587137 T1156 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 587145 T1156 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 587146 T1156 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 587159 T1156 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 587160 T1156 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 587263 T1156 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 587264 T1156 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 587267 T1156 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 587267 T1156 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 587278 T1156 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 587278 T1156 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 587281 T1156 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 587282 T1156 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 587285 T1156 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 587285 T1156 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 587288 T1156 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 587289 T1156 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 587296 T1156 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 587297 T1156 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 587557 T1156 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 587560 T1156 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63741
[junit4] 2> 587561 T1156 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 587561 T1156 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 587562 T1156 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374567225751
[junit4] 2> 587562 T1156 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374567225751/'
[junit4] 2> 587597 T1156 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374567225751/solr.xml
[junit4] 2> 587682 T1156 oasc.CoreContainer.<init> New CoreContainer 1751868799
[junit4] 2> 587683 T1156 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374567225751/]
[junit4] 2> 587684 T1156 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 587685 T1156 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 587685 T1156 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 587686 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 587686 T1156 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 587687 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 587687 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 587687 T1156 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 587688 T1156 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 587688 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 587705 T1156 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 587705 T1156 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:30042/solr
[junit4] 2> 587706 T1156 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 587707 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 587713 T1176 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27e9bf53 name:ZooKeeperConnection Watcher:127.0.0.1:30042 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 587714 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 587717 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 587727 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 587730 T1178 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@561c6ffd name:ZooKeeperConnection Watcher:127.0.0.1:30042/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 587730 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 587732 T1156 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 587742 T1156 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 587746 T1156 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 587753 T1156 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63741_c_%2Fuk
[junit4] 2> 587755 T1156 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63741_c_%2Fuk
[junit4] 2> 587759 T1156 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 587778 T1156 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 587781 T1156 oasc.Overseer.start Overseer (id=90083637686763523-127.0.0.1:63741_c_%2Fuk-n_0000000000) starting
[junit4] 2> 587794 T1156 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 587805 T1180 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 587806 T1156 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 587809 T1156 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 587811 T1156 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 587814 T1179 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 587820 T1181 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 587820 T1181 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 587822 T1181 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 589319 T1179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 589320 T1179 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:63741_c_%2Fuk",
[junit4] 2> "base_url":"http://127.0.0.1:63741/c_/uk"}
[junit4] 2> 589321 T1179 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4] 2> 589321 T1179 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 589338 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 589824 T1181 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 589824 T1181 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374567225751/collection1
[junit4] 2> 589824 T1181 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 589825 T1181 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 589826 T1181 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 589827 T1181 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374567225751/collection1/'
[junit4] 2> 589828 T1181 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374567225751/collection1/lib/README' to classloader
[junit4] 2> 589829 T1181 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374567225751/collection1/lib/classes/' to classloader
[junit4] 2> 589881 T1181 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 589944 T1181 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 590046 T1181 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 590056 T1181 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 590625 T1181 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 590637 T1181 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 590641 T1181 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 590657 T1181 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 590662 T1181 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 590666 T1181 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 590668 T1181 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 590668 T1181 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 590668 T1181 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 590670 T1181 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 590670 T1181 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 590670 T1181 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 590671 T1181 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374567225751/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/control/data/
[junit4] 2> 590671 T1181 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4494356f
[junit4] 2> 590673 T1181 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/control/data
[junit4] 2> 590674 T1181 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/control/data/index/
[junit4] 2> 590674 T1181 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 590675 T1181 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/control/data/index
[junit4] 2> 590679 T1181 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a7c25fb lockFactory=org.apache.lucene.store.NativeFSLockFactory@5347e91c),segFN=segments_1,generation=1}
[junit4] 2> 590680 T1181 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 590683 T1181 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 590683 T1181 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 590684 T1181 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 590685 T1181 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 590685 T1181 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 590685 T1181 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 590686 T1181 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 590686 T1181 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 590686 T1181 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 590703 T1181 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 590710 T1181 oass.SolrIndexSearcher.<init> Opening Searcher@43a205cc main
[junit4] 2> 590712 T1181 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 590712 T1181 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 590715 T1182 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@43a205cc main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 590716 T1181 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 590717 T1181 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63741/c_/uk collection:control_collection shard:shard1
[junit4] 2> 590717 T1181 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 590733 T1181 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 590753 T1181 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 590756 T1181 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 590756 T1181 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 590756 T1181 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63741/c_/uk/collection1/
[junit4] 2> 590757 T1181 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 590757 T1181 oasc.SyncStrategy.syncToMe http://127.0.0.1:63741/c_/uk/collection1/ has no replicas
[junit4] 2> 590757 T1181 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63741/c_/uk/collection1/ shard1
[junit4] 2> 590758 T1181 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 590843 T1179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 590859 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 590867 T1181 oasc.ZkController.register We are http://127.0.0.1:63741/c_/uk/collection1/ and leader is http://127.0.0.1:63741/c_/uk/collection1/
[junit4] 2> 590867 T1181 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63741/c_/uk
[junit4] 2> 590867 T1181 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 590867 T1181 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 590868 T1181 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 590869 T1181 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 590870 T1156 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 590871 T1156 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 590871 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 590882 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4] 2> 590886 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 590891 T1185 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2108a072 name:ZooKeeperConnection Watcher:127.0.0.1:30042/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 590891 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 590893 T1156 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 590896 T1156 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 591165 T1156 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 591168 T1156 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26506
[junit4] 2> 591169 T1156 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 591169 T1156 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 591170 T1156 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374567229346
[junit4] 2> 591170 T1156 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374567229346/'
[junit4] 2> 591205 T1156 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374567229346/solr.xml
[junit4] 2> 591290 T1156 oasc.CoreContainer.<init> New CoreContainer 433139994
[junit4] 2> 591291 T1156 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374567229346/]
[junit4] 2> 591292 T1156 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 591293 T1156 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 591293 T1156 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 591294 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 591294 T1156 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 591295 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 591295 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 591295 T1156 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 591296 T1156 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 591296 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 591314 T1156 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 591315 T1156 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:30042/solr
[junit4] 2> 591315 T1156 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 591316 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 591323 T1196 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c6807ce name:ZooKeeperConnection Watcher:127.0.0.1:30042 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 591324 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 591347 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 591360 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 591362 T1198 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2dca4c76 name:ZooKeeperConnection Watcher:127.0.0.1:30042/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 591362 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 591375 T1156 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 592366 T1179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 592367 T1179 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:63741_c_%2Fuk",
[junit4] 2> "base_url":"http://127.0.0.1:63741/c_/uk"}
[junit4] 2> 592383 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 592383 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 592383 T1156 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26506_c_%2Fuk
[junit4] 2> 592383 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 592386 T1156 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26506_c_%2Fuk
[junit4] 2> 592389 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 592389 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 592390 T1198 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 592390 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 592391 T1178 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 592391 T1185 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 592403 T1199 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 592403 T1199 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 592406 T1199 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 593889 T1179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 593890 T1179 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:26506_c_%2Fuk",
[junit4] 2> "base_url":"http://127.0.0.1:26506/c_/uk"}
[junit4] 2> 593890 T1179 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4] 2> 593891 T1179 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 593905 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 593905 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 593905 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 594408 T1199 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 594408 T1199 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374567229346/collection1
[junit4] 2> 594408 T1199 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 594409 T1199 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 594409 T1199 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 594411 T1199 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374567229346/collection1/'
[junit4] 2> 594412 T1199 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374567229346/collection1/lib/README' to classloader
[junit4] 2> 594413 T1199 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374567229346/collection1/lib/classes/' to classloader
[junit4] 2> 594465 T1199 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 594528 T1199 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 594629 T1199 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 594639 T1199 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 595214 T1199 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 595226 T1199 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 595230 T1199 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 595247 T1199 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 595252 T1199 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 595258 T1199 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 595259 T1199 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 595260 T1199 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 595260 T1199 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 595261 T1199 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 595261 T1199 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 595262 T1199 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 595262 T1199 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374567229346/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty1/
[junit4] 2> 595262 T1199 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4494356f
[junit4] 2> 595265 T1199 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty1
[junit4] 2> 595266 T1199 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty1/index/
[junit4] 2> 595266 T1199 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 595266 T1199 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty1/index
[junit4] 2> 595270 T1199 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6ed0ccc2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2af9862d),segFN=segments_1,generation=1}
[junit4] 2> 595271 T1199 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 595275 T1199 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 595275 T1199 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 595276 T1199 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 595276 T1199 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 595277 T1199 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 595277 T1199 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 595278 T1199 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 595278 T1199 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 595279 T1199 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 595296 T1199 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 595304 T1199 oass.SolrIndexSearcher.<init> Opening Searcher@11eabd93 main
[junit4] 2> 595305 T1199 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 595306 T1199 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 595309 T1200 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11eabd93 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 595311 T1199 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 595311 T1199 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26506/c_/uk collection:collection1 shard:shard1
[junit4] 2> 595311 T1199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 595331 T1199 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 595348 T1199 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 595359 T1199 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 595360 T1199 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 595360 T1199 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:26506/c_/uk/collection1/
[junit4] 2> 595360 T1199 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 595361 T1199 oasc.SyncStrategy.syncToMe http://127.0.0.1:26506/c_/uk/collection1/ has no replicas
[junit4] 2> 595361 T1199 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:26506/c_/uk/collection1/ shard1
[junit4] 2> 595361 T1199 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 595410 T1179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 595426 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 595426 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 595426 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 595477 T1199 oasc.ZkController.register We are http://127.0.0.1:26506/c_/uk/collection1/ and leader is http://127.0.0.1:26506/c_/uk/collection1/
[junit4] 2> 595477 T1199 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26506/c_/uk
[junit4] 2> 595477 T1199 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 595477 T1199 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 595478 T1199 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 595480 T1199 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 595481 T1156 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 595482 T1156 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 595482 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 595770 T1156 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 595773 T1156 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:42351
[junit4] 2> 595773 T1156 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 595774 T1156 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 595774 T1156 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374567233944
[junit4] 2> 595775 T1156 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374567233944/'
[junit4] 2> 595810 T1156 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374567233944/solr.xml
[junit4] 2> 595897 T1156 oasc.CoreContainer.<init> New CoreContainer 2006674892
[junit4] 2> 595898 T1156 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374567233944/]
[junit4] 2> 595900 T1156 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 595900 T1156 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 595901 T1156 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 595901 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 595902 T1156 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 595902 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 595902 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 595903 T1156 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 595903 T1156 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 595904 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 595921 T1156 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 595922 T1156 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:30042/solr
[junit4] 2> 595923 T1156 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 595924 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 595952 T1212 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27b95600 name:ZooKeeperConnection Watcher:127.0.0.1:30042 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 595952 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 595994 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 596007 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 596013 T1214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6075debf name:ZooKeeperConnection Watcher:127.0.0.1:30042/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 596014 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 596032 T1156 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 596941 T1179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 596942 T1179 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:26506_c_%2Fuk",
[junit4] 2> "base_url":"http://127.0.0.1:26506/c_/uk"}
[junit4] 2> 596965 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 596966 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 596965 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 596965 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 597035 T1156 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42351_c_%2Fuk
[junit4] 2> 597037 T1156 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42351_c_%2Fuk
[junit4] 2> 597040 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 597040 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 597040 T1214 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 597040 T1198 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 597040 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 597041 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 597042 T1178 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 597042 T1185 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 597057 T1215 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 597058 T1215 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 597059 T1215 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 598477 T1179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 598478 T1179 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42351_c_%2Fuk",
[junit4] 2> "base_url":"http://127.0.0.1:42351/c_/uk"}
[junit4] 2> 598479 T1179 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 598479 T1179 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 598497 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 598497 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 598497 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 598497 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 599062 T1215 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 599062 T1215 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374567233944/collection1
[junit4] 2> 599062 T1215 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 599063 T1215 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 599064 T1215 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 599065 T1215 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374567233944/collection1/'
[junit4] 2> 599066 T1215 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374567233944/collection1/lib/README' to classloader
[junit4] 2> 599067 T1215 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374567233944/collection1/lib/classes/' to classloader
[junit4] 2> 599125 T1215 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 599183 T1215 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 599285 T1215 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 599295 T1215 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 599854 T1215 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 599862 T1215 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 599864 T1215 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 599878 T1215 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 599882 T1215 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 599886 T1215 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 599887 T1215 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 599887 T1215 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 599888 T1215 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 599889 T1215 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 599889 T1215 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 599889 T1215 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 599889 T1215 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374567233944/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty2/
[junit4] 2> 599889 T1215 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4494356f
[junit4] 2> 599891 T1215 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty2
[junit4] 2> 599892 T1215 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty2/index/
[junit4] 2> 599892 T1215 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 599892 T1215 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty2/index
[junit4] 2> 599895 T1215 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@36e6f68d lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fb4bac6),segFN=segments_1,generation=1}
[junit4] 2> 599895 T1215 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 599898 T1215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 599898 T1215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 599899 T1215 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 599899 T1215 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 599900 T1215 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 599900 T1215 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 599900 T1215 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 599900 T1215 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 599901 T1215 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 599911 T1215 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 599916 T1215 oass.SolrIndexSearcher.<init> Opening Searcher@713c9f08 main
[junit4] 2> 599917 T1215 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 599918 T1215 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 599920 T1216 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@713c9f08 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 599921 T1215 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 599922 T1215 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42351/c_/uk collection:collection1 shard:shard2
[junit4] 2> 599922 T1215 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 599934 T1215 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 599953 T1215 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 599963 T1215 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 599963 T1215 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 599964 T1215 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42351/c_/uk/collection1/
[junit4] 2> 599964 T1215 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 599964 T1215 oasc.SyncStrategy.syncToMe http://127.0.0.1:42351/c_/uk/collection1/ has no replicas
[junit4] 2> 599964 T1215 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42351/c_/uk/collection1/ shard2
[junit4] 2> 599964 T1215 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 600002 T1179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 600018 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 600018 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 600018 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 600018 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 600034 T1215 oasc.ZkController.register We are http://127.0.0.1:42351/c_/uk/collection1/ and leader is http://127.0.0.1:42351/c_/uk/collection1/
[junit4] 2> 600034 T1215 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42351/c_/uk
[junit4] 2> 600034 T1215 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 600034 T1215 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 600035 T1215 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 600036 T1215 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 600037 T1156 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 600038 T1156 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 600038 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 600227 T1156 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 600229 T1156 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38974
[junit4] 2> 600229 T1156 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 600230 T1156 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 600230 T1156 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374567238496
[junit4] 2> 600230 T1156 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374567238496/'
[junit4] 2> 600253 T1156 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374567238496/solr.xml
[junit4] 2> 600309 T1156 oasc.CoreContainer.<init> New CoreContainer 849026707
[junit4] 2> 600310 T1156 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374567238496/]
[junit4] 2> 600311 T1156 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 600311 T1156 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 600311 T1156 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 600312 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 600312 T1156 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 600312 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 600313 T1156 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 600313 T1156 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 600313 T1156 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 600313 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 600325 T1156 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 600326 T1156 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:30042/solr
[junit4] 2> 600326 T1156 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 600327 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 600330 T1228 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f59a3a5 name:ZooKeeperConnection Watcher:127.0.0.1:30042 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 600330 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 600347 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 600360 T1156 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 600362 T1230 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6789e638 name:ZooKeeperConnection Watcher:127.0.0.1:30042/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 600363 T1156 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 600375 T1156 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 601379 T1156 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38974_c_%2Fuk
[junit4] 2> 601384 T1156 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38974_c_%2Fuk
[junit4] 2> 601387 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 601387 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 601388 T1214 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 601388 T1230 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 601388 T1198 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 601388 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 601389 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 601390 T1185 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 601390 T1178 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 601407 T1231 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 601407 T1231 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 601408 T1231 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 601525 T1179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 601526 T1179 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42351_c_%2Fuk",
[junit4] 2> "base_url":"http://127.0.0.1:42351/c_/uk"}
[junit4] 2> 601531 T1179 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:38974_c_%2Fuk",
[junit4] 2> "base_url":"http://127.0.0.1:38974/c_/uk"}
[junit4] 2> 601532 T1179 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 601532 T1179 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 601548 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 601548 T1230 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 601548 T1185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 601548 T1198 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 601548 T1178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 602410 T1231 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 602410 T1231 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374567238496/collection1
[junit4] 2> 602410 T1231 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 602411 T1231 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 602411 T1231 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 602413 T1231 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374567238496/collection1/'
[junit4] 2> 602414 T1231 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374567238496/collection1/lib/classes/' to classloader
[junit4] 2> 602415 T1231 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374567238496/collection1/lib/README' to classloader
[junit4] 2> 602467 T1231 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 602525 T1231 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 602626 T1231 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 602636 T1231 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 603209 T1231 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 603227 T1231 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 603231 T1231 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 603247 T1231 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 603252 T1231 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 603257 T1231 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 603259 T1231 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 603259 T1231 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 603259 T1231 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 603261 T1231 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 603261 T1231 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 603261 T1231 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 603262 T1231 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374567238496/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty3/
[junit4] 2> 603262 T1231 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4494356f
[junit4] 2> 603264 T1231 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty3
[junit4] 2> 603265 T1231 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty3/index/
[junit4] 2> 603265 T1231 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 603266 T1231 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374567225438/jetty3/index
[junit4] 2> 603269 T1231 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4cb4ec73 lockFactory=org.apache.lucene.store.NativeFSLockFactory@74647510),segFN=segments_1,generation=1}
[junit4] 2> 603270 T1231 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 603274 T1231 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 603274 T1231 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 603275 T1231 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 603275 T1231 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 603276 T1231 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 603276 T1231 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 603276 T1231 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 603277 T1231 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 603277 T1231 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 603294 T1231 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 603301 T1231 oass.SolrIndexSearcher.<init> Opening Searcher@7b4f42b main
[junit4] 2> 603302 T1231 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 603303 T1231 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 603306 T1232 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b4f42b main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 603308 T1231 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 603308 T1231 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38974/c_/uk collection:collection1 shard:shard1
[junit4] 2> 603308 T1231 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 603330 T1231 oasc.ZkController.register We are http://127.0.0.1:38974/c_/uk/collection1/ and leader is http://127.0.0.1:26506/c_/uk/collection1/
[junit4] 2> 603330 T1231 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38974/c_/uk
[junit4] 2> 603330 T1231 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 603330 T1231 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C661 name=collection1 org.apache.solr.core.SolrCore@2b4568e5 url=http://127.0.0.1:38974/c_/uk/collection1 node=127.0.0.1:38974_c_%2Fuk C661_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:38974_c_%2Fuk, base_url=http://127.0.0.1:38974/c_/uk}
[junit4] 2> 603331 T1233 C661 P38974 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 603331 T1233 C661 P38974 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 603331 T1231 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 603332 T1233 C661 P38974 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 603332 T1233 C661 P38974 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 603333 T1156 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 603333 T1156 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 603334 T1233 C661 P38974 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 603334 T1156 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 603351 T1188 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 603617 T1156 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 603620 T1156 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:52668
[junit4] 2> 603620 T1156 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 603621 T1156 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 603621 T1156 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1374567241800
[junit4] 2> 603622 T1156 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1374567241800/'
[junit4] 2> 603657 T1156 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1374567241800/solr.xml
[junit4] 2> 603742 T1156 oasc.CoreContainer.<init> New CoreContainer 449200358
[junit4] 2> 603743 T1156 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1374567241800/]
[junit4] 2> 603745 T1156 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 603745 T1156 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 603745 T1156 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 603746 T1156 oashc.HttpShardHandlerFactory.getParameter Se
[...truncated too long message...]
rg.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4] 2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_MX, timezone=Africa/Blantyre
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=159114760,total=358547456
[junit4] 2> NOTE: All tests run in this JVM: [AutoCommitTest, IndexSchemaTest, AnalysisAfterCoreReloadTest, FileBasedSpellCheckerTest, FastVectorHighlighterTest, TestSolrCoreProperties, BlockDirectoryTest, TestFastLRUCache, TestSolrQueryParser, TestSolrDeletionPolicy2, TestSolrQueryParserDefaultOperatorResource, TestCopyFieldCollectionResource, QueryEqualityTest, ScriptEngineTest, RegexBoostProcessorTest, CacheHeaderTest, TestJoin, TestSolrQueryParserResource, CoreAdminHandlerTest, UUIDFieldTest, LukeRequestHandlerTest, TestConfig, TestStressVersions, TestPseudoReturnFields, TestSolrIndexConfig, PrimitiveFieldTypeTest, TestShardHandlerFactory, BinaryUpdateRequestHandlerTest, TestLRUCache, TestSchemaVersionResource, ZkControllerTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestFaceting, TestQuerySenderListener, TestDocSet, SuggesterTest, AliasIntegrationTest, OverseerCollectionProcessorTest, TestPHPSerializedResponseWriter, TestAddFieldRealTimeGet, SynonymTokenizerTest, SuggesterFSTTest, SearchHandlerTest, PreAnalyzedFieldTest, SpatialFilterTest, SolrTestCaseJ4Test, LoggingHandlerTest, TestValueSourceCache, SolrCoreTest, ShowFileRequestHandlerTest, BadComponentTest, SchemaVersionSpecificBehaviorTest, TestSurroundQueryParser, PrimUtilsTest, DocValuesTest, MBeansHandlerTest, StressHdfsTest, RAMDirectoryFactoryTest, TestSerializedLuceneMatchVersion, SystemInfoHandlerTest, DocumentBuilderTest, TestCoreContainer, SliceStateTest, ResourceLoaderTest, BadCopyFieldTest, TestStressRecovery, StatelessScriptUpdateProcessorFactoryTest, PluginInfoTest, SpellingQueryConverterTest, AssignTest, TestFieldTypeCollectionResource, TestDynamicFieldCollectionResource, TestGroupingSearch, TestSearchPerf, ResponseLogComponentTest, ClusterStateUpdateTest, RecoveryZkTest, TermsComponentTest, BasicDistributedZkTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=C4391FDD5D0E0708 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_MX -Dtests.timezone=Africa/Blantyre -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4] > 1) Thread[id=1665, name=recoveryCmdExecutor-792-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4] > at __randomizedtesting.SeedInfo.seed([C4391FDD5D0E0708]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=1665, name=recoveryCmdExecutor-792-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4] > at __randomizedtesting.SeedInfo.seed([C4391FDD5D0E0708]:0)
[junit4] Completed on J0 in 416.32s, 1 test, 2 errors <<< FAILURES!
[...truncated 549 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:396: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:369: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1247: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:890: There were test failures: 318 suites, 1342 tests, 2 suite-level errors, 671 ignored (4 assumptions)
Total time: 90 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 331 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/331/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=3445, name=recoveryCmdExecutor-1703-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=3445, name=recoveryCmdExecutor-1703-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
at __randomizedtesting.SeedInfo.seed([6C00272C40A9A4C]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3445, name=recoveryCmdExecutor-1703-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3445, name=recoveryCmdExecutor-1703-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
at __randomizedtesting.SeedInfo.seed([6C00272C40A9A4C]:0)
Build Log:
[...truncated 9782 lines...]
[junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4] 2> 1368643 T2935 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /pqj/
[junit4] 2> 1368647 T2935 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1374822399322
[junit4] 2> 1368649 T2935 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 1368649 T2936 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1368750 T2935 oasc.ZkTestServer.run start zk server on port:42819
[junit4] 2> 1368751 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1368755 T2942 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78b09077 name:ZooKeeperConnection Watcher:127.0.0.1:42819 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1368755 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1368756 T2935 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1368767 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1368769 T2944 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@103c587d name:ZooKeeperConnection Watcher:127.0.0.1:42819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1368769 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1368770 T2935 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1368780 T2935 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1368782 T2935 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1368785 T2935 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1368793 T2935 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1368793 T2935 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1368798 T2935 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1368798 T2935 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1368901 T2935 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1368902 T2935 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1368905 T2935 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1368905 T2935 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1368908 T2935 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1368908 T2935 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1368911 T2935 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1368911 T2935 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1368914 T2935 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1368914 T2935 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1368922 T2935 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1368923 T2935 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1368925 T2935 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1368926 T2935 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1369214 T2935 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1369219 T2935 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:42822
[junit4] 2> 1369220 T2935 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1369221 T2935 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1369221 T2935 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374822399609
[junit4] 2> 1369221 T2935 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374822399609/'
[junit4] 2> 1369260 T2935 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374822399609/solr.xml
[junit4] 2> 1369348 T2935 oasc.CoreContainer.<init> New CoreContainer 1554702447
[junit4] 2> 1369349 T2935 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374822399609/]
[junit4] 2> 1369350 T2935 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1369351 T2935 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1369351 T2935 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1369352 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1369352 T2935 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1369352 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1369353 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1369353 T2935 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1369354 T2935 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1369354 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1369372 T2935 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42819/solr
[junit4] 2> 1369372 T2935 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1369373 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1369376 T2955 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3100f3da name:ZooKeeperConnection Watcher:127.0.0.1:42819 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1369376 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1369387 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1369399 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1369402 T2957 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c8e6e24 name:ZooKeeperConnection Watcher:127.0.0.1:42819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1369402 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1369409 T2935 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1369417 T2935 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1369419 T2935 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1369426 T2935 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42822_pqj
[junit4] 2> 1369428 T2935 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42822_pqj
[junit4] 2> 1369430 T2935 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1369443 T2935 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1369445 T2935 oasc.Overseer.start Overseer (id=90100360762294275-127.0.0.1:42822_pqj-n_0000000000) starting
[junit4] 2> 1369455 T2935 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1369465 T2959 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1369465 T2935 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1369467 T2935 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1369475 T2935 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1369477 T2958 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1369480 T2960 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1369480 T2960 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1369482 T2960 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1370980 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1370982 T2958 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42822_pqj",
[junit4] 2> "base_url":"http://127.0.0.1:42822/pqj"}
[junit4] 2> 1370982 T2958 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4] 2> 1370982 T2958 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1370998 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1371484 T2960 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1371484 T2960 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374822399609/collection1
[junit4] 2> 1371484 T2960 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1371485 T2960 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1371485 T2960 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1371486 T2960 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374822399609/collection1/'
[junit4] 2> 1371488 T2960 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374822399609/collection1/lib/classes/' to classloader
[junit4] 2> 1371488 T2960 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374822399609/collection1/lib/README' to classloader
[junit4] 2> 1371549 T2960 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1371608 T2960 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1371709 T2960 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1371719 T2960 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1372309 T2960 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1372321 T2960 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1372325 T2960 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1372338 T2960 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1372343 T2960 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1372347 T2960 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1372348 T2960 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1372349 T2960 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1372349 T2960 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1372350 T2960 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1372350 T2960 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1372351 T2960 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1372351 T2960 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374822399609/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/control/data/
[junit4] 2> 1372351 T2960 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@547e97d8
[junit4] 2> 1372352 T2960 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/control/data
[junit4] 2> 1372352 T2960 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/control/data/index/
[junit4] 2> 1372353 T2960 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 1372353 T2960 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/control/data/index
[junit4] 2> 1372358 T2960 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@74b719fe lockFactory=org.apache.lucene.store.NativeFSLockFactory@216e7f30),segFN=segments_1,generation=1}
[junit4] 2> 1372358 T2960 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1372363 T2960 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1372363 T2960 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1372364 T2960 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1372365 T2960 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1372365 T2960 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1372365 T2960 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1372366 T2960 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1372366 T2960 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1372367 T2960 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1372384 T2960 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1372391 T2960 oass.SolrIndexSearcher.<init> Opening Searcher@652f24f5 main
[junit4] 2> 1372393 T2960 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1372393 T2960 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1372397 T2961 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@652f24f5 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1372399 T2960 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1372400 T2960 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42822/pqj collection:control_collection shard:shard1
[junit4] 2> 1372400 T2960 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 1372419 T2960 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1372438 T2960 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1372450 T2960 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1372450 T2960 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1372450 T2960 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42822/pqj/collection1/
[junit4] 2> 1372450 T2960 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1372451 T2960 oasc.SyncStrategy.syncToMe http://127.0.0.1:42822/pqj/collection1/ has no replicas
[junit4] 2> 1372451 T2960 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42822/pqj/collection1/ shard1
[junit4] 2> 1372451 T2960 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 1372502 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1372516 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1372567 T2960 oasc.ZkController.register We are http://127.0.0.1:42822/pqj/collection1/ and leader is http://127.0.0.1:42822/pqj/collection1/
[junit4] 2> 1372567 T2960 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42822/pqj
[junit4] 2> 1372567 T2960 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1372567 T2960 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 1372567 T2960 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1372569 T2960 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1372570 T2935 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 1372571 T2935 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1372572 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1372584 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4] 2> 1372586 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1372588 T2964 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17bd8e5e name:ZooKeeperConnection Watcher:127.0.0.1:42819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1372588 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1372589 T2935 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1372591 T2935 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 1372868 T2935 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1372871 T2935 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:42826
[junit4] 2> 1372872 T2935 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1372872 T2935 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1372873 T2935 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374822403265
[junit4] 2> 1372873 T2935 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374822403265/'
[junit4] 2> 1372912 T2935 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374822403265/solr.xml
[junit4] 2> 1372999 T2935 oasc.CoreContainer.<init> New CoreContainer 996517898
[junit4] 2> 1373000 T2935 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374822403265/]
[junit4] 2> 1373002 T2935 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1373002 T2935 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1373002 T2935 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1373003 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1373003 T2935 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1373003 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1373004 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1373004 T2935 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1373005 T2935 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1373005 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1373022 T2935 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42819/solr
[junit4] 2> 1373023 T2935 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1373024 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1373027 T2975 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3219a20d name:ZooKeeperConnection Watcher:127.0.0.1:42819 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1373027 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1373041 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1373053 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1373055 T2977 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@265f918 name:ZooKeeperConnection Watcher:127.0.0.1:42819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1373056 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1373066 T2935 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1374026 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1374027 T2958 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42822_pqj",
[junit4] 2> "base_url":"http://127.0.0.1:42822/pqj"}
[junit4] 2> 1374041 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1374041 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1374041 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1374068 T2935 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42826_pqj
[junit4] 2> 1374070 T2935 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42826_pqj
[junit4] 2> 1374072 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1374072 T2957 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 1374072 T2977 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 1374072 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1374072 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1374073 T2964 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 1374087 T2978 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1374087 T2978 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1374089 T2978 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1375545 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1375546 T2958 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42826_pqj",
[junit4] 2> "base_url":"http://127.0.0.1:42826/pqj"}
[junit4] 2> 1375546 T2958 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4] 2> 1375547 T2958 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1375562 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1375562 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1375562 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1376091 T2978 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1376091 T2978 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374822403265/collection1
[junit4] 2> 1376091 T2978 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1376092 T2978 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1376092 T2978 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1376093 T2978 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374822403265/collection1/'
[junit4] 2> 1376095 T2978 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374822403265/collection1/lib/README' to classloader
[junit4] 2> 1376095 T2978 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374822403265/collection1/lib/classes/' to classloader
[junit4] 2> 1376149 T2978 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1376207 T2978 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1376308 T2978 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1376319 T2978 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1376764 T2978 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1376776 T2978 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1376780 T2978 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1376794 T2978 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1376799 T2978 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1376804 T2978 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1376805 T2978 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1376805 T2978 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1376805 T2978 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1376806 T2978 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1376806 T2978 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1376807 T2978 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1376807 T2978 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374822403265/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty1/
[junit4] 2> 1376807 T2978 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@547e97d8
[junit4] 2> 1376808 T2978 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty1
[junit4] 2> 1376809 T2978 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty1/index/
[junit4] 2> 1376809 T2978 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 1376809 T2978 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty1/index
[junit4] 2> 1376813 T2978 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5fd5b7f8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e756029),segFN=segments_1,generation=1}
[junit4] 2> 1376813 T2978 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1376819 T2978 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1376819 T2978 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1376820 T2978 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1376821 T2978 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1376821 T2978 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1376822 T2978 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1376822 T2978 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1376823 T2978 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1376823 T2978 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1376840 T2978 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1376848 T2978 oass.SolrIndexSearcher.<init> Opening Searcher@358df8de main
[junit4] 2> 1376849 T2978 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1376849 T2978 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1376855 T2979 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@358df8de main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1376857 T2978 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1376858 T2978 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42826/pqj collection:collection1 shard:shard1
[junit4] 2> 1376858 T2978 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 1376875 T2978 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1376906 T2978 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1376916 T2978 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1376916 T2978 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1376916 T2978 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42826/pqj/collection1/
[junit4] 2> 1376916 T2978 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1376917 T2978 oasc.SyncStrategy.syncToMe http://127.0.0.1:42826/pqj/collection1/ has no replicas
[junit4] 2> 1376917 T2978 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42826/pqj/collection1/ shard1
[junit4] 2> 1376917 T2978 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1377066 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1377081 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1377081 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1377081 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1377133 T2978 oasc.ZkController.register We are http://127.0.0.1:42826/pqj/collection1/ and leader is http://127.0.0.1:42826/pqj/collection1/
[junit4] 2> 1377133 T2978 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42826/pqj
[junit4] 2> 1377133 T2978 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1377133 T2978 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 1377134 T2978 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1377135 T2978 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1377136 T2935 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 1377137 T2935 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1377138 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1377406 T2935 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1377409 T2935 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:42829
[junit4] 2> 1377409 T2935 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1377410 T2935 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1377410 T2935 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374822407823
[junit4] 2> 1377411 T2935 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374822407823/'
[junit4] 2> 1377446 T2935 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374822407823/solr.xml
[junit4] 2> 1377527 T2935 oasc.CoreContainer.<init> New CoreContainer 1227320939
[junit4] 2> 1377528 T2935 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374822407823/]
[junit4] 2> 1377529 T2935 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1377530 T2935 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1377530 T2935 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1377530 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1377531 T2935 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1377531 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1377531 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1377532 T2935 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1377532 T2935 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1377533 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1377549 T2935 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42819/solr
[junit4] 2> 1377549 T2935 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1377550 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1377553 T2991 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@466008ac name:ZooKeeperConnection Watcher:127.0.0.1:42819 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1377554 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1377568 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1377580 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1377582 T2993 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6efcc134 name:ZooKeeperConnection Watcher:127.0.0.1:42819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1377582 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1377591 T2935 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1378594 T2935 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42829_pqj
[junit4] 2> 1378595 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1378596 T2935 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42829_pqj
[junit4] 2> 1378596 T2958 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42826_pqj",
[junit4] 2> "base_url":"http://127.0.0.1:42826/pqj"}
[junit4] 2> 1378612 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1378613 T2977 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1378613 T2993 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1378613 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1378614 T2964 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1378621 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1378621 T2993 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1378621 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1378623 T2957 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1378624 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1378624 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1378627 T2994 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1378627 T2994 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1378629 T2994 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1380125 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1380126 T2958 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42829_pqj",
[junit4] 2> "base_url":"http://127.0.0.1:42829/pqj"}
[junit4] 2> 1380126 T2958 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 1380126 T2958 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1380140 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1380141 T2993 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1380141 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1380140 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1380631 T2994 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1380631 T2994 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374822407823/collection1
[junit4] 2> 1380631 T2994 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1380632 T2994 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1380632 T2994 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1380633 T2994 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374822407823/collection1/'
[junit4] 2> 1380635 T2994 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374822407823/collection1/lib/classes/' to classloader
[junit4] 2> 1380635 T2994 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374822407823/collection1/lib/README' to classloader
[junit4] 2> 1380689 T2994 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1380747 T2994 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1380848 T2994 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1380859 T2994 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1381444 T2994 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1381456 T2994 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1381466 T2994 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1381479 T2994 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1381483 T2994 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1381488 T2994 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1381488 T2994 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1381489 T2994 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1381489 T2994 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1381490 T2994 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1381490 T2994 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1381490 T2994 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1381490 T2994 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374822407823/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty2/
[junit4] 2> 1381491 T2994 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@547e97d8
[junit4] 2> 1381492 T2994 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty2
[junit4] 2> 1381492 T2994 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty2/index/
[junit4] 2> 1381492 T2994 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 1381493 T2994 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty2/index
[junit4] 2> 1381496 T2994 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6df0430e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fca331e),segFN=segments_1,generation=1}
[junit4] 2> 1381497 T2994 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1381502 T2994 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1381502 T2994 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1381503 T2994 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1381504 T2994 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1381504 T2994 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1381505 T2994 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1381505 T2994 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1381506 T2994 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1381506 T2994 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1381522 T2994 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1381530 T2994 oass.SolrIndexSearcher.<init> Opening Searcher@3f12384e main
[junit4] 2> 1381531 T2994 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1381531 T2994 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1381536 T2995 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f12384e main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1381538 T2994 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1381539 T2994 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42829/pqj collection:collection1 shard:shard2
[junit4] 2> 1381539 T2994 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 1381556 T2994 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1381584 T2994 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 1381594 T2994 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1381594 T2994 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1381594 T2994 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42829/pqj/collection1/
[junit4] 2> 1381595 T2994 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1381595 T2994 oasc.SyncStrategy.syncToMe http://127.0.0.1:42829/pqj/collection1/ has no replicas
[junit4] 2> 1381595 T2994 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42829/pqj/collection1/ shard2
[junit4] 2> 1381595 T2994 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1381645 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1381659 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1381659 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1381659 T2993 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1381659 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1381709 T2994 oasc.ZkController.register We are http://127.0.0.1:42829/pqj/collection1/ and leader is http://127.0.0.1:42829/pqj/collection1/
[junit4] 2> 1381709 T2994 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42829/pqj
[junit4] 2> 1381709 T2994 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1381709 T2994 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 1381709 T2994 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1381711 T2994 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1381712 T2935 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 1381713 T2935 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1381713 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1382002 T2935 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1382005 T2935 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:42832
[junit4] 2> 1382006 T2935 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1382006 T2935 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1382006 T2935 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374822412399
[junit4] 2> 1382007 T2935 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374822412399/'
[junit4] 2> 1382043 T2935 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374822412399/solr.xml
[junit4] 2> 1382138 T2935 oasc.CoreContainer.<init> New CoreContainer 1101369824
[junit4] 2> 1382139 T2935 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374822412399/]
[junit4] 2> 1382141 T2935 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1382141 T2935 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1382141 T2935 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1382142 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1382142 T2935 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1382143 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1382143 T2935 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1382143 T2935 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1382144 T2935 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1382144 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1382165 T2935 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42819/solr
[junit4] 2> 1382165 T2935 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1382166 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1382194 T3007 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c1587a4 name:ZooKeeperConnection Watcher:127.0.0.1:42819 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1382194 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1382196 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1382208 T2935 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1382210 T3009 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7fba5f1c name:ZooKeeperConnection Watcher:127.0.0.1:42819/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1382210 T2935 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1382214 T2935 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1383164 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1383165 T2958 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42829_pqj",
[junit4] 2> "base_url":"http://127.0.0.1:42829/pqj"}
[junit4] 2> 1383178 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1383178 T3009 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1383178 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1383178 T2993 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1383178 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1383216 T2935 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42832_pqj
[junit4] 2> 1383217 T2935 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42832_pqj
[junit4] 2> 1383219 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1383219 T2993 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1383219 T3009 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1383219 T2977 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1383219 T2957 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1383220 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1383220 T2964 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1383220 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1383221 T2993 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1383222 T3009 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1383228 T3010 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1383228 T3010 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1383230 T3010 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1384682 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1384683 T2958 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42832_pqj",
[junit4] 2> "base_url":"http://127.0.0.1:42832/pqj"}
[junit4] 2> 1384683 T2958 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 1384684 T2958 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1384700 T2977 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1384700 T2993 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1384700 T3009 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1384700 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1384700 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1385232 T3010 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1385232 T3010 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374822412399/collection1
[junit4] 2> 1385232 T3010 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1385233 T3010 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1385233 T3010 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1385234 T3010 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374822412399/collection1/'
[junit4] 2> 1385236 T3010 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374822412399/collection1/lib/README' to classloader
[junit4] 2> 1385236 T3010 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374822412399/collection1/lib/classes/' to classloader
[junit4] 2> 1385291 T3010 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1385349 T3010 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1385450 T3010 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1385461 T3010 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1386046 T3010 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1386057 T3010 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1386061 T3010 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1386073 T3010 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1386078 T3010 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1386081 T3010 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1386082 T3010 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1386083 T3010 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1386083 T3010 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1386084 T3010 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1386084 T3010 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1386084 T3010 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1386084 T3010 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374822412399/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty3/
[junit4] 2> 1386085 T3010 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@547e97d8
[junit4] 2> 1386086 T3010 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty3
[junit4] 2> 1386086 T3010 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty3/index/
[junit4] 2> 1386086 T3010 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 1386086 T3010 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374822399322/jetty3/index
[junit4] 2> 1386090 T3010 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@48140b1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@795e0ac1),segFN=segments_1,generation=1}
[junit4] 2> 1386090 T3010 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1386095 T3010 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1386096 T3010 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1386096 T3010 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1386097 T3010 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1386098 T3010 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1386098 T3010 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1386098 T3010 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1386099 T3010 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1386099 T3010 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1386116 T3010 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1386123 T3010 oass.SolrIndexSearcher.<init> Opening Searcher@7d349410 main
[junit4] 2> 1386125 T3010 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1386125 T3010 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1386130 T3011 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d349410 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1386132 T3010 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1386133 T3010 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42832/pqj collection:collection1 shard:shard1
[junit4] 2> 1386133 T3010 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 1386152 T3010 oasc.ZkController.register We are http://127.0.0.1:42832/pqj/collection1/ and leader is http://127.0.0.1:42826/pqj/collection1/
[junit4] 2> 1386152 T3010 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42832/pqj
[junit4] 2> 1386152 T3010 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 1386153 T3010 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C984 name=collection1 org.apache.solr.core.SolrCore@73e94795 url=http://127.0.0.1:42832/pqj/collection1 node=127.0.0.1:42832_pqj C984_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:42832_pqj, base_url=http://127.0.0.1:42832/pqj}
[junit4] 2> 1386153 T3012 C984 P42832 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 1386153 T3010 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1386155 T3012 C984 P42832 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 1386155 T3012 C984 P42832 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 1386155 T3012 C984 P42832 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1386156 T2935 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 1386157 T2935 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1386157 T2935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1386207 T3012 C984 P42832 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1386208 T2958 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1386209 T2958 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"recovering",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42832_pqj",
[junit4] 2> "base_url":"http://127.0.0.1:42832/pqj"}
[junit4] 2> 1386220 T2967 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 1386297 T2957 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurre
[...truncated too long message...]
), multiDefault=PostingsFormat(name=Lucene41WithOrds), n_tf1=Pulsing41(freqCutoff=20 minBlockSize=99 maxBlockSize=225), n_dt1=PostingsFormat(name=Lucene41WithOrds), genre_s=PostingsFormat(name=Asserting), author_t=PostingsFormat(name=Lucene41WithOrds), range_facet_l=Pulsing41(freqCutoff=20 minBlockSize=99 maxBlockSize=225), n_ti1=PostingsFormat(name=MockSep), text=PostingsFormat(name=Asserting), _version_=Pulsing41(freqCutoff=20 minBlockSize=99 maxBlockSize=225), val_i=PostingsFormat(name=MockSep), SubjectTerms_mfacet=Pulsing41(freqCutoff=20 minBlockSize=99 maxBlockSize=225), series_t=PostingsFormat(name=Lucene41WithOrds), a_t=Pulsing41(freqCutoff=20 minBlockSize=99 maxBlockSize=225), n_tdt1=Pulsing41(freqCutoff=20 minBlockSize=99 maxBlockSize=225), regex_dup_A_s=PostingsFormat(name=Asserting), price=PostingsFormat(name=Asserting), other_tl1=Pulsing41(freqCutoff=20 minBlockSize=99 maxBlockSize=225), n_l1=Pulsing41(freqCutoff=20 minBlockSize=99 maxBlockSize=225), a_si=PostingsFormat(name=Lucene41WithOrds), inStock=PostingsFormat(name=Asserting)}, docValues:{timestamp=DocValuesFormat(name=CheapBastard)}, sim=DefaultSimilarity, locale=ar_SY, timezone=America/Dawson_Creek
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=2,free=89224440,total=376700928
[junit4] 2> NOTE: All tests run in this JVM: [TestFastWriter, TestFaceting, UnloadDistributedZkTest, TestStressVersions, SearchHandlerTest, JsonLoaderTest, TestLFUCache, TestPHPSerializedResponseWriter, TestFastLRUCache, FieldMutatingUpdateProcessorTest, ReturnFieldsTest, TestGroupingSearch, TestSolrXmlPersistence, DistributedQueryElevationComponentTest, DocumentBuilderTest, AlternateDirectoryTest, TestFunctionQuery, SynonymTokenizerTest, ShowFileRequestHandlerTest, TestPropInjectDefaults, TestFieldTypeResource, ClusterStateUpdateTest, TestSolrQueryParser, SpatialFilterTest, TestFieldResource, ZkControllerTest, TestSolrCoreProperties, TestRandomDVFaceting, BadComponentTest, DirectUpdateHandlerOptimizeTest, TestSearchPerf, TestPhraseSuggestions, TestMaxScoreQueryParser, TestSchemaVersionResource, ResourceLoaderTest, PeerSyncTest, MBeansHandlerTest, RecoveryZkTest, TestDefaultSearchFieldResource, DocValuesTest, TestSolrQueryParserResource, TestBadConfig, PrimitiveFieldTypeTest, QueryElevationComponentTest, LoggingHandlerTest, IndexSchemaTest, AnalysisAfterCoreReloadTest, SolrIndexSplitterTest, TestShardHandlerFactory, SchemaVersionSpecificBehaviorTest, PrimUtilsTest, SolrTestCaseJ4Test, TestFuzzyAnalyzedSuggestions, SolrCoreTest, ScriptEngineTest, TestCopyFieldCollectionResource, TestXIncludeConfig, SuggesterTest, AutoCommitTest, TestLRUCache, SoftAutoCommitTest, TestReplicationHandler, DirectSolrConnectionTest, TestExtendedDismaxParser, TestSerializedLuceneMatchVersion, ExternalFileFieldSortTest, TestAddFieldRealTimeGet, BasicFunctionalityTest, LukeRequestHandlerTest, OverseerTest, QueryParsingTest, UpdateRequestProcessorFactoryTest, StatelessScriptUpdateProcessorFactoryTest, CircularListTest, TestPluginEnable, LegacyHTMLStripCharFilterTest, TestSolrIndexConfig, BasicDistributedZkTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=6C00272C40A9A4C -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_SY -Dtests.timezone=America/Dawson_Creek -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4] > 1) Thread[id=3445, name=recoveryCmdExecutor-1703-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4] > at __randomizedtesting.SeedInfo.seed([6C00272C40A9A4C]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=3445, name=recoveryCmdExecutor-1703-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4] > at __randomizedtesting.SeedInfo.seed([6C00272C40A9A4C]:0)
[junit4] Completed on J0 in 416.31s, 1 test, 2 errors <<< FAILURES!
[...truncated 483 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:396: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:369: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1230: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:873: There were test failures: 318 suites, 1342 tests, 2 suite-level errors, 654 ignored (4 assumptions)
Total time: 86 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 330 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/330/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=3003, name=recoveryCmdExecutor-1116-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=3003, name=recoveryCmdExecutor-1116-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([254F2F6795AF1A1F]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3003, name=recoveryCmdExecutor-1116-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3003, name=recoveryCmdExecutor-1116-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([254F2F6795AF1A1F]:0)
Build Log:
[...truncated 9869 lines...]
[junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4] 2> 1319446 T2490 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /yiz/
[junit4] 2> 1319451 T2490 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1374738258793
[junit4] 2> 1319453 T2490 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 1319454 T2491 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1319555 T2490 oasc.ZkTestServer.run start zk server on port:38445
[junit4] 2> 1319556 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1319560 T2497 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56870785 name:ZooKeeperConnection Watcher:127.0.0.1:38445 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1319560 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1319560 T2490 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1319581 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1319583 T2499 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7bf646a1 name:ZooKeeperConnection Watcher:127.0.0.1:38445/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1319583 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1319583 T2490 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1319594 T2490 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1319597 T2490 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1319600 T2490 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1319608 T2490 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1319609 T2490 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1319622 T2490 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1319623 T2490 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1319726 T2490 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1319727 T2490 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1319730 T2490 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1319731 T2490 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1319742 T2490 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1319742 T2490 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1319745 T2490 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1319746 T2490 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1319749 T2490 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1319750 T2490 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1319753 T2490 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1319754 T2490 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1319757 T2490 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1319758 T2490 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1320032 T2490 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1320036 T2490 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38448
[junit4] 2> 1320037 T2490 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1320037 T2490 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1320038 T2490 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374738259110
[junit4] 2> 1320038 T2490 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374738259110/'
[junit4] 2> 1320075 T2490 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374738259110/solr.xml
[junit4] 2> 1320160 T2490 oasc.CoreContainer.<init> New CoreContainer 993750239
[junit4] 2> 1320161 T2490 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374738259110/]
[junit4] 2> 1320163 T2490 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1320163 T2490 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1320164 T2490 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1320164 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1320165 T2490 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1320165 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1320166 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1320166 T2490 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1320167 T2490 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1320167 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1320185 T2490 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38445/solr
[junit4] 2> 1320185 T2490 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1320186 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1320189 T2510 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42afada0 name:ZooKeeperConnection Watcher:127.0.0.1:38445 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1320190 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1320202 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1320214 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1320216 T2512 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cb91d29 name:ZooKeeperConnection Watcher:127.0.0.1:38445/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1320216 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1320223 T2490 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1320228 T2490 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1320237 T2490 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1320239 T2490 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38448_yiz
[junit4] 2> 1320240 T2490 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38448_yiz
[junit4] 2> 1320248 T2490 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1320261 T2490 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1320268 T2490 oasc.Overseer.start Overseer (id=90094846528782339-127.0.0.1:38448_yiz-n_0000000000) starting
[junit4] 2> 1320273 T2490 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1320286 T2514 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1320287 T2490 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1320289 T2490 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1320296 T2490 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1320299 T2513 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1320305 T2515 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1320305 T2515 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1320306 T2515 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1321803 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1321804 T2513 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:38448_yiz",
[junit4] 2> "base_url":"http://127.0.0.1:38448/yiz"}
[junit4] 2> 1321804 T2513 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4] 2> 1321804 T2513 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1321820 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1322309 T2515 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1322309 T2515 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374738259110/collection1
[junit4] 2> 1322309 T2515 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1322310 T2515 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1322311 T2515 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1322312 T2515 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374738259110/collection1/'
[junit4] 2> 1322313 T2515 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374738259110/collection1/lib/README' to classloader
[junit4] 2> 1322314 T2515 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374738259110/collection1/lib/classes/' to classloader
[junit4] 2> 1322367 T2515 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1322426 T2515 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1322528 T2515 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1322538 T2515 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1323115 T2515 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1323127 T2515 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1323131 T2515 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1323145 T2515 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1323150 T2515 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1323155 T2515 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1323156 T2515 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1323156 T2515 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1323156 T2515 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1323157 T2515 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1323158 T2515 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1323158 T2515 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1323158 T2515 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1374738259110/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/control/data/
[junit4] 2> 1323159 T2515 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@199ee8a8
[junit4] 2> 1323160 T2515 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/control/data
[junit4] 2> 1323160 T2515 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/control/data/index/
[junit4] 2> 1323160 T2515 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 1323162 T2515 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/control/data/index
[junit4] 2> 1323194 T2515 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/index1011510138tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@f292bb7),segFN=segments_1,generation=1}
[junit4] 2> 1323195 T2515 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1323199 T2515 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1323200 T2515 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1323200 T2515 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1323201 T2515 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1323202 T2515 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1323202 T2515 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1323202 T2515 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1323203 T2515 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1323203 T2515 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1323220 T2515 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1323229 T2515 oass.SolrIndexSearcher.<init> Opening Searcher@116e5955 main
[junit4] 2> 1323230 T2515 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1323231 T2515 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1323234 T2516 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@116e5955 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1323235 T2515 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1323236 T2515 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38448/yiz collection:control_collection shard:shard1
[junit4] 2> 1323236 T2515 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 1323254 T2515 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1323271 T2515 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1323282 T2515 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1323282 T2515 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1323283 T2515 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38448/yiz/collection1/
[junit4] 2> 1323283 T2515 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1323283 T2515 oasc.SyncStrategy.syncToMe http://127.0.0.1:38448/yiz/collection1/ has no replicas
[junit4] 2> 1323283 T2515 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38448/yiz/collection1/ shard1
[junit4] 2> 1323284 T2515 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 1323325 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1323346 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1323402 T2515 oasc.ZkController.register We are http://127.0.0.1:38448/yiz/collection1/ and leader is http://127.0.0.1:38448/yiz/collection1/
[junit4] 2> 1323402 T2515 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38448/yiz
[junit4] 2> 1323402 T2515 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1323402 T2515 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 1323403 T2515 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1323405 T2515 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1323406 T2490 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 1323406 T2490 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1323407 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1323419 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4] 2> 1323420 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1323422 T2519 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17746af0 name:ZooKeeperConnection Watcher:127.0.0.1:38445/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1323423 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1323424 T2490 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1323426 T2490 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 1323688 T2490 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1323691 T2490 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38452
[junit4] 2> 1323692 T2490 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1323692 T2490 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1323693 T2490 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374738262767
[junit4] 2> 1323693 T2490 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374738262767/'
[junit4] 2> 1323730 T2490 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374738262767/solr.xml
[junit4] 2> 1323814 T2490 oasc.CoreContainer.<init> New CoreContainer 421993604
[junit4] 2> 1323815 T2490 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374738262767/]
[junit4] 2> 1323817 T2490 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1323817 T2490 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1323818 T2490 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1323818 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1323819 T2490 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1323819 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1323820 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1323820 T2490 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1323821 T2490 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1323821 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1323838 T2490 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38445/solr
[junit4] 2> 1323839 T2490 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1323840 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1323843 T2530 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ead4929 name:ZooKeeperConnection Watcher:127.0.0.1:38445 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1323843 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1323845 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1323858 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1323860 T2532 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@427ab5a1 name:ZooKeeperConnection Watcher:127.0.0.1:38445/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1323860 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1323872 T2490 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1324852 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1324854 T2513 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:38448_yiz",
[junit4] 2> "base_url":"http://127.0.0.1:38448/yiz"}
[junit4] 2> 1324858 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1324858 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1324858 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1324875 T2490 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38452_yiz
[junit4] 2> 1324877 T2490 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38452_yiz
[junit4] 2> 1324879 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1324879 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1324880 T2519 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 1324880 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1324881 T2532 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 1324881 T2512 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 1324888 T2533 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1324889 T2533 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1324890 T2533 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1326372 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1326374 T2513 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:38452_yiz",
[junit4] 2> "base_url":"http://127.0.0.1:38452/yiz"}
[junit4] 2> 1326374 T2513 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4] 2> 1326374 T2513 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1326388 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1326388 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1326388 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1326893 T2533 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1326893 T2533 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374738262767/collection1
[junit4] 2> 1326893 T2533 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1326894 T2533 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1326894 T2533 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1326895 T2533 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374738262767/collection1/'
[junit4] 2> 1326897 T2533 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374738262767/collection1/lib/README' to classloader
[junit4] 2> 1326897 T2533 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374738262767/collection1/lib/classes/' to classloader
[junit4] 2> 1326951 T2533 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1327010 T2533 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1327111 T2533 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1327121 T2533 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1327696 T2533 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1327708 T2533 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1327711 T2533 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1327725 T2533 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1327730 T2533 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1327734 T2533 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1327735 T2533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1327736 T2533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1327736 T2533 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1327737 T2533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1327737 T2533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1327738 T2533 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1327738 T2533 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1374738262767/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty1/
[junit4] 2> 1327738 T2533 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@199ee8a8
[junit4] 2> 1327739 T2533 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty1
[junit4] 2> 1327739 T2533 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty1/index/
[junit4] 2> 1327740 T2533 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 1327742 T2533 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty1/index
[junit4] 2> 1327749 T2533 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/index1011510139tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c4a66b9),segFN=segments_1,generation=1}
[junit4] 2> 1327750 T2533 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1327755 T2533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1327755 T2533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1327756 T2533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1327756 T2533 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1327757 T2533 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1327757 T2533 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1327757 T2533 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1327758 T2533 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1327758 T2533 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1327775 T2533 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1327783 T2533 oass.SolrIndexSearcher.<init> Opening Searcher@6003e06d main
[junit4] 2> 1327784 T2533 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1327784 T2533 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1327789 T2534 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6003e06d main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1327791 T2533 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1327791 T2533 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38452/yiz collection:collection1 shard:shard1
[junit4] 2> 1327791 T2533 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 1327808 T2533 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1327824 T2533 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1327833 T2533 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1327834 T2533 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1327834 T2533 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38452/yiz/collection1/
[junit4] 2> 1327834 T2533 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1327834 T2533 oasc.SyncStrategy.syncToMe http://127.0.0.1:38452/yiz/collection1/ has no replicas
[junit4] 2> 1327835 T2533 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38452/yiz/collection1/ shard1
[junit4] 2> 1327835 T2533 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1327894 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1327910 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1327910 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1327910 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1327953 T2533 oasc.ZkController.register We are http://127.0.0.1:38452/yiz/collection1/ and leader is http://127.0.0.1:38452/yiz/collection1/
[junit4] 2> 1327953 T2533 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38452/yiz
[junit4] 2> 1327953 T2533 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1327953 T2533 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 1327954 T2533 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1327956 T2533 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1327957 T2490 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 1327957 T2490 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1327958 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1328239 T2490 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1328243 T2490 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38455
[junit4] 2> 1328244 T2490 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1328244 T2490 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1328245 T2490 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374738267309
[junit4] 2> 1328245 T2490 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374738267309/'
[junit4] 2> 1328282 T2490 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374738267309/solr.xml
[junit4] 2> 1328368 T2490 oasc.CoreContainer.<init> New CoreContainer 1441817408
[junit4] 2> 1328369 T2490 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374738267309/]
[junit4] 2> 1328371 T2490 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1328371 T2490 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1328372 T2490 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1328372 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1328373 T2490 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1328373 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1328374 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1328374 T2490 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1328374 T2490 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1328375 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1328391 T2490 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38445/solr
[junit4] 2> 1328391 T2490 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1328392 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1328421 T2546 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f22bd6b name:ZooKeeperConnection Watcher:127.0.0.1:38445 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1328422 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1328424 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1328434 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1328436 T2548 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@89e92fc name:ZooKeeperConnection Watcher:127.0.0.1:38445/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1328437 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1328448 T2490 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1329424 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1329426 T2513 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:38452_yiz",
[junit4] 2> "base_url":"http://127.0.0.1:38452/yiz"}
[junit4] 2> 1329437 T2548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1329437 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1329437 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1329437 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1329451 T2490 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38455_yiz
[junit4] 2> 1329453 T2490 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38455_yiz
[junit4] 2> 1329456 T2548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1329456 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1329456 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1329456 T2519 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1329457 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1329458 T2548 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1329459 T2532 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1329459 T2512 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1329466 T2549 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1329466 T2549 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1329467 T2549 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1330943 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1330945 T2513 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:38455_yiz",
[junit4] 2> "base_url":"http://127.0.0.1:38455/yiz"}
[junit4] 2> 1330945 T2513 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 1330945 T2513 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1330951 T2548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1330951 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1330951 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1330951 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1331471 T2549 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1331471 T2549 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374738267309/collection1
[junit4] 2> 1331471 T2549 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1331472 T2549 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1331473 T2549 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1331474 T2549 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374738267309/collection1/'
[junit4] 2> 1331475 T2549 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374738267309/collection1/lib/README' to classloader
[junit4] 2> 1331476 T2549 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374738267309/collection1/lib/classes/' to classloader
[junit4] 2> 1331529 T2549 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1331591 T2549 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1331693 T2549 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1331704 T2549 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1332338 T2549 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1332350 T2549 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1332354 T2549 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1332368 T2549 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1332373 T2549 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1332378 T2549 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1332379 T2549 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1332379 T2549 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1332379 T2549 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1332380 T2549 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1332381 T2549 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1332381 T2549 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1332382 T2549 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1374738267309/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty2/
[junit4] 2> 1332382 T2549 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@199ee8a8
[junit4] 2> 1332383 T2549 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty2
[junit4] 2> 1332383 T2549 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty2/index/
[junit4] 2> 1332383 T2549 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 1332385 T2549 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty2/index
[junit4] 2> 1332392 T2549 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/index1011510140tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@53214248),segFN=segments_1,generation=1}
[junit4] 2> 1332392 T2549 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1332397 T2549 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1332398 T2549 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1332398 T2549 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1332399 T2549 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1332400 T2549 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1332400 T2549 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1332400 T2549 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1332401 T2549 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1332401 T2549 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1332419 T2549 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1332428 T2549 oass.SolrIndexSearcher.<init> Opening Searcher@3ba1b77c main
[junit4] 2> 1332429 T2549 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1332430 T2549 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1332434 T2550 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ba1b77c main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1332436 T2549 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1332436 T2549 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38455/yiz collection:collection1 shard:shard2
[junit4] 2> 1332436 T2549 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 1332454 T2549 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1332535 T2549 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 1332597 T2549 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1332597 T2549 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1332597 T2549 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38455/yiz/collection1/
[junit4] 2> 1332598 T2549 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1332598 T2549 oasc.SyncStrategy.syncToMe http://127.0.0.1:38455/yiz/collection1/ has no replicas
[junit4] 2> 1332598 T2549 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38455/yiz/collection1/ shard2
[junit4] 2> 1332598 T2549 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1334036 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1334054 T2548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1334055 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1334055 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1334054 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1334097 T2549 oasc.ZkController.register We are http://127.0.0.1:38455/yiz/collection1/ and leader is http://127.0.0.1:38455/yiz/collection1/
[junit4] 2> 1334097 T2549 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38455/yiz
[junit4] 2> 1334097 T2549 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1334098 T2549 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 1334098 T2549 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1334101 T2549 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1334102 T2490 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 1334103 T2490 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1334103 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1334377 T2490 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1334381 T2490 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:51578
[junit4] 2> 1334381 T2490 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1334382 T2490 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1334382 T2490 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374738273456
[junit4] 2> 1334383 T2490 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374738273456/'
[junit4] 2> 1334422 T2490 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374738273456/solr.xml
[junit4] 2> 1334513 T2490 oasc.CoreContainer.<init> New CoreContainer 1510273587
[junit4] 2> 1334514 T2490 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374738273456/]
[junit4] 2> 1334516 T2490 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1334516 T2490 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1334517 T2490 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1334517 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1334518 T2490 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1334518 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1334519 T2490 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1334519 T2490 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1334520 T2490 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1334520 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1334538 T2490 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38445/solr
[junit4] 2> 1334538 T2490 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1334539 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1334542 T2562 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27657383 name:ZooKeeperConnection Watcher:127.0.0.1:38445 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1334543 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1334545 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1334557 T2490 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1334559 T2564 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30302867 name:ZooKeeperConnection Watcher:127.0.0.1:38445/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1334559 T2490 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1334572 T2490 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1335561 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1335563 T2513 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:38455_yiz",
[junit4] 2> "base_url":"http://127.0.0.1:38455/yiz"}
[junit4] 2> 1335589 T2490 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51578_yiz
[junit4] 2> 1335605 T2490 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51578_yiz
[junit4] 2> 1335606 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1335606 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1335606 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1335606 T2564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1335606 T2548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1335617 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1335617 T2519 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1335618 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1335618 T2548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1335618 T2564 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1335618 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1335619 T2532 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1335620 T2548 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1335620 T2512 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 1335626 T2565 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1335626 T2565 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1335628 T2565 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1337119 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1337121 T2513 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:51578_yiz",
[junit4] 2> "base_url":"http://127.0.0.1:51578/yiz"}
[junit4] 2> 1337121 T2513 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 1337121 T2513 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1337138 T2548 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1337138 T2532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1337138 T2512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1337138 T2564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1337138 T2519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1337632 T2565 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1337632 T2565 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374738273456/collection1
[junit4] 2> 1337632 T2565 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1337633 T2565 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1337634 T2565 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1337635 T2565 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374738273456/collection1/'
[junit4] 2> 1337636 T2565 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374738273456/collection1/lib/classes/' to classloader
[junit4] 2> 1337637 T2565 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374738273456/collection1/lib/README' to classloader
[junit4] 2> 1337680 T2565 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1337719 T2565 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1337820 T2565 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1337831 T2565 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1338437 T2565 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1338449 T2565 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1338452 T2565 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1338465 T2565 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1338470 T2565 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1338474 T2565 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1338475 T2565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1338475 T2565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1338476 T2565 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1338476 T2565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1338477 T2565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1338477 T2565 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1338478 T2565 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1374738273456/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty3/
[junit4] 2> 1338478 T2565 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@199ee8a8
[junit4] 2> 1338479 T2565 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty3
[junit4] 2> 1338479 T2565 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty3/index/
[junit4] 2> 1338479 T2565 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 1338481 T2565 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1374738258793/jetty3/index
[junit4] 2> 1338488 T2565 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/index1011510141tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@76ce5486),segFN=segments_1,generation=1}
[junit4] 2> 1338488 T2565 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1338493 T2565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1338493 T2565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1338494 T2565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1338495 T2565 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1338495 T2565 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1338495 T2565 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1338496 T2565 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1338496 T2565 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1338497 T2565 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1338513 T2565 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1338521 T2565 oass.SolrIndexSearcher.<init> Opening Searcher@8ef2bf5 main
[junit4] 2> 1338522 T2565 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1338523 T2565 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1338526 T2566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8ef2bf5 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1338528 T2565 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1338529 T2565 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51578/yiz collection:collection1 shard:shard1
[junit4] 2> 1338529 T2565 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 1338548 T2565 oasc.ZkController.register We are http://127.0.0.1:51578/yiz/collection1/ and leader is http://127.0.0.1:38452/yiz/collection1/
[junit4] 2> 1338549 T2565 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51578/yiz
[junit4] 2> 1338549 T2565 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 1338549 T2565 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C669 name=collection1 org.apache.solr.core.SolrCore@4efc8ba4 url=http://127.0.0.1:51578/yiz/collection1 node=127.0.0.1:51578_yiz C669_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:51578_yiz, base_url=http://127.0.0.1:51578/yiz}
[junit4] 2> 1338549 T2567 C669 P51578 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 1338550 T2565 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1338550 T2567 C669 P51578 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 1338551 T2567 C669 P51578 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 1338551 T2567 C669 P51578 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1338551 T2490 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 1338552 T2490 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1338552 T2490 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1338552 T2567 C669 P51578 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1338562 T2522 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 1338643 T2513 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1338644 T2513 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"recovering"
[...truncated too long message...]
ze=128), sequence_i=Lucene41(blocksize=128), name=Pulsing41(freqCutoff=5 minBlockSize=74 maxBlockSize=170), foo_i=PostingsFormat(name=TestBloomFilteredLucene41Postings), regex_dup_B_s=PostingsFormat(name=Direct), multiDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tf1=PostingsFormat(name=Direct), n_dt1=PostingsFormat(name=TestBloomFilteredLucene41Postings), genre_s=Lucene41(blocksize=128), author_t=PostingsFormat(name=TestBloomFilteredLucene41Postings), range_facet_l=PostingsFormat(name=Direct), n_ti1=Pulsing41(freqCutoff=5 minBlockSize=74 maxBlockSize=170), text=Lucene41(blocksize=128), _version_=PostingsFormat(name=Direct), val_i=Pulsing41(freqCutoff=5 minBlockSize=74 maxBlockSize=170), SubjectTerms_mfacet=PostingsFormat(name=Direct), series_t=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_t=PostingsFormat(name=Direct), n_tdt1=PostingsFormat(name=Direct), regex_dup_A_s=Lucene41(blocksize=128), price=Lucene41(blocksize=128), other_tl1=PostingsFormat(name=Direct), n_l1=PostingsFormat(name=Direct), a_si=PostingsFormat(name=TestBloomFilteredLucene41Postings), inStock=Lucene41(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Disk)}, sim=DefaultSimilarity, locale=et, timezone=Europe/Mariehamn
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=224758160,total=320339968
[junit4] 2> NOTE: All tests run in this JVM: [TestSurroundQueryParser, CacheHeaderTest, SuggesterFSTTest, FastVectorHighlighterTest, TestBadConfig, TestSolrXml, TestPerFieldSimilarity, TestIBSimilarityFactory, ScriptEngineTest, FileBasedSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestDocSet, RecoveryZkTest, SpatialFilterTest, PrimUtilsTest, TestShardHandlerFactory, TestStressRecovery, TestConfig, TestGroupingSearch, ChaosMonkeyNothingIsSafeTest, SearchHandlerTest, TestFieldResource, UpdateRequestProcessorFactoryTest, JsonLoaderTest, AnalysisAfterCoreReloadTest, SliceStateUpdateTest, TestXIncludeConfig, DistributedQueryElevationComponentTest, ParsingFieldUpdateProcessorsTest, TestPluginEnable, TestQuerySenderListener, ResourceLoaderTest, TestSerializedLuceneMatchVersion, ClusterStateUpdateTest, InfoHandlerTest, SimpleFacetsTest, TestLRUCache, PreAnalyzedFieldTest, TestFieldTypeCollectionResource, TestFunctionQuery, TestFieldTypeResource, TestSolrIndexConfig, TestSolrDeletionPolicy1, AddSchemaFieldsUpdateProcessorFactoryTest, SuggesterTest, TestPhraseSuggestions, DistributedSpellCheckComponentTest, TestSchemaVersionResource, TestSchemaNameResource, BadComponentTest, QueryEqualityTest, TestRandomDVFaceting, TestWriterPerf, TestCharFilters, ZkCLITest, SyncSliceTest, TermsComponentTest, TestSort, LukeRequestHandlerTest, CoreAdminHandlerTest, OverseerCollectionProcessorTest, UUIDFieldTest, QueryParsingTest, SolrTestCaseJ4Test, TestValueSourceCache, MinimalSchemaTest, PolyFieldTest, WordBreakSolrSpellCheckerTest, ReturnFieldsTest, AliasIntegrationTest, StressHdfsTest, AutoCommitTest, SpellingQueryConverterTest, MoreLikeThisHandlerTest, TestFastWriter, TestCopyFieldCollectionResource, ShardRoutingTest, BasicDistributedZkTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=254F2F6795AF1A1F -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=et -Dtests.timezone=Europe/Mariehamn -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4] > 1) Thread[id=3003, name=recoveryCmdExecutor-1116-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4] > at __randomizedtesting.SeedInfo.seed([254F2F6795AF1A1F]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=3003, name=recoveryCmdExecutor-1116-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:291)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4] > at __randomizedtesting.SeedInfo.seed([254F2F6795AF1A1F]:0)
[junit4] Completed on J0 in 531.70s, 1 test, 2 errors <<< FAILURES!
[...truncated 481 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:396: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:369: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1230: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:873: There were test failures: 318 suites, 1342 tests, 2 suite-level errors, 567 ignored (3 assumptions)
Total time: 84 minutes 0 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure