You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/01/03 17:29:06 UTC
[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 484 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/484/
1 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest
Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=476, name=coreLoadExecutor-217-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=476, name=coreLoadExecutor-217-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
at __randomizedtesting.SeedInfo.seed([DAC754BB48CAF363]:0)
at java.lang.Thread.getStackTrace(Thread.java:1568)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:551)
at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1035)
at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:933)
at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:925)
at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:350)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:254)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:468)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:759)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:628)
at org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:223)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:592)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:258)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:250)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.AssertionError: Directory not closed: BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@2e1c837f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51fd1e7f)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
... 1 more
Build Log:
[...truncated 10012 lines...]
[junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4] 2> 169940 T374 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 169950 T374 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./solrtest-RecoveryZkTest-1388763205832
[junit4] 2> 169951 T374 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 169952 T375 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 170053 T374 oasc.ZkTestServer.run start zk server on port:39030
[junit4] 2> 170055 T374 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 170080 T382 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c9b58e2 name:ZooKeeperConnection Watcher:127.0.0.1:39030 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 170081 T374 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 170082 T374 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 170096 T374 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 170100 T385 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73a57187 name:ZooKeeperConnection Watcher:127.0.0.1:39030/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 170100 T374 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 170101 T374 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 170108 T374 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 170114 T374 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 170119 T374 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 170126 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 170128 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 170145 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 170147 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 170256 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 170258 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 170269 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 170271 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 170280 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 170282 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 170290 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 170292 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 170301 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 170303 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 170311 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 170313 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 170321 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 170322 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 170331 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 170333 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 170347 T374 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 170348 T374 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 171211 T374 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 171218 T374 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:39040
[junit4] 2> 171220 T374 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 171220 T374 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 171221 T374 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.RecoveryZkTest-controljetty-1388763206235
[junit4] 2> 171222 T374 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.RecoveryZkTest-controljetty-1388763206235/'
[junit4] 2> 171348 T374 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1388763206235/solr.xml
[junit4] 2> 171618 T374 oasc.CoreContainer.<init> New CoreContainer 1136577864
[junit4] 2> 171618 T374 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.RecoveryZkTest-controljetty-1388763206235/]
[junit4] 2> 171621 T374 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 171621 T374 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 171622 T374 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 171622 T374 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 171623 T374 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 171623 T374 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 171623 T374 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 171624 T374 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 171624 T374 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 171640 T374 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 171641 T374 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 171642 T374 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 171642 T374 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39030/solr
[junit4] 2> 171643 T374 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 171644 T374 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 171668 T397 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1050b243 name:ZooKeeperConnection Watcher:127.0.0.1:39030 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 171669 T374 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 171675 T374 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 171696 T400 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@459c1eef name:ZooKeeperConnection Watcher:127.0.0.1:39030/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 171697 T374 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 171703 T374 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 171711 T374 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 171721 T374 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 171785 T374 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39040_
[junit4] 2> 171787 T374 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39040_
[junit4] 2> 171829 T374 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 171835 T374 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 171864 T374 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 171870 T374 oasc.Overseer.start Overseer (id=91013985457864707-127.0.0.1:39040_-n_0000000000) starting
[junit4] 2> 171877 T374 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 171885 T402 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 171886 T374 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 171891 T374 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 171895 T374 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 171902 T401 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 171915 T403 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 171916 T403 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 171918 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 171919 T403 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 171922 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 171923 T401 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39040",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:39040_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 171924 T401 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 171943 T401 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 171950 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 171952 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 172920 T403 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 172920 T403 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.RecoveryZkTest-controljetty-1388763206235/collection1
[junit4] 2> 172920 T403 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 172921 T403 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 172921 T403 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 172923 T403 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.RecoveryZkTest-controljetty-1388763206235/collection1/'
[junit4] 2> 172924 T403 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1388763206235/collection1/lib/README' to classloader
[junit4] 2> 172925 T403 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1388763206235/collection1/lib/classes/' to classloader
[junit4] 2> 172994 T403 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 173053 T403 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 173155 T403 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 173170 T403 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 173721 T403 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 173724 T403 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 173727 T403 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 173732 T403 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 173773 T403 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 173773 T403 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.RecoveryZkTest-controljetty-1388763206235/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1388763205831/control/data/
[junit4] 2> 173774 T403 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31714e34
[junit4] 2> 173774 T403 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/control/data
[junit4] 2> 173775 T403 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1388763205831/control/data/index/
[junit4] 2> 173775 T403 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1388763205831/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 173775 T403 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/control/data/index
[junit4] 2> 173776 T403 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=13, maxMergedSegmentMB=61.82421875, floorSegmentMB=1.9921875, forceMergeDeletesPctAllowed=20.459119156739835, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6258587176718375
[junit4] 2> 173777 T403 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c41070b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6e8378c2),segFN=segments_1,generation=1}
[junit4] 2> 173777 T403 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 173782 T403 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 173782 T403 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 173783 T403 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 173783 T403 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 173783 T403 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 173783 T403 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 173784 T403 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 173784 T403 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 173784 T403 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 173785 T403 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 173785 T403 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 173786 T403 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 173786 T403 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 173786 T403 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 173787 T403 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 173787 T403 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 173788 T403 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 173803 T403 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 173809 T403 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 173809 T403 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 173810 T403 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=53.09375, floorSegmentMB=1.7001953125, forceMergeDeletesPctAllowed=9.78988450627622, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21570963271496765
[junit4] 2> 173810 T403 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c41070b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6e8378c2),segFN=segments_1,generation=1}
[junit4] 2> 173811 T403 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 173811 T403 oass.SolrIndexSearcher.<init> Opening Searcher@5392025d main
[junit4] 2> 173814 T404 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5392025d main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 173817 T403 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 173818 T374 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 173819 T374 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 173818 T407 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39040 collection:control_collection shard:shard1
[junit4] 2> 173821 T374 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 173822 T407 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 173824 T410 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ced7f7 name:ZooKeeperConnection Watcher:127.0.0.1:39030/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 173824 T374 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 173826 T374 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 173838 T374 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 173847 T407 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 173850 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 173851 T407 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 173851 T407 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C309 name=collection1 org.apache.solr.core.SolrCore@6f4ec2e4 url=http://127.0.0.1:39040/collection1 node=127.0.0.1:39040_ C309_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:39040, core=collection1, node_name=127.0.0.1:39040_}
[junit4] 2> 173851 T407 C309 P39040 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39040/collection1/
[junit4] 2> 173852 T407 C309 P39040 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 173852 T407 C309 P39040 oasc.SyncStrategy.syncToMe http://127.0.0.1:39040/collection1/ has no replicas
[junit4] 2> 173852 T407 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39040/collection1/ shard1
[junit4] 2> 173852 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 173852 T407 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 173862 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 173871 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 173871 T410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 173878 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 173882 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 173985 T410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 173985 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 174029 T407 oasc.ZkController.register We are http://127.0.0.1:39040/collection1/ and leader is http://127.0.0.1:39040/collection1/
[junit4] 2> 174029 T407 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39040
[junit4] 2> 174029 T407 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 174029 T407 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 174029 T407 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 174031 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 174031 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 174031 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 174032 T407 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 174033 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 174034 T401 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:39040",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:39040_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 174038 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 174109 T374 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 174112 T374 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:39052
[junit4] 2> 174113 T374 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 174113 T374 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 174114 T374 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1388763209719
[junit4] 2> 174114 T374 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1388763209719/'
[junit4] 2> 174142 T410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 174142 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 174150 T374 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1388763209719/solr.xml
[junit4] 2> 174242 T374 oasc.CoreContainer.<init> New CoreContainer 761251357
[junit4] 2> 174242 T374 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1388763209719/]
[junit4] 2> 174244 T374 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 174244 T374 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 174245 T374 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 174245 T374 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 174245 T374 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 174246 T374 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 174246 T374 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 174246 T374 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 174247 T374 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 174257 T374 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 174258 T374 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 174258 T374 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 174259 T374 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39030/solr
[junit4] 2> 174259 T374 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 174260 T374 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 174263 T422 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@704e5b5d name:ZooKeeperConnection Watcher:127.0.0.1:39030 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 174264 T374 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 174268 T374 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 174275 T425 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@209b324 name:ZooKeeperConnection Watcher:127.0.0.1:39030/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 174275 T374 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 174282 T374 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 175287 T374 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39052_
[junit4] 2> 175318 T374 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39052_
[junit4] 2> 175325 T400 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 175326 T425 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 175325 T410 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 175349 T426 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 175349 T426 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 175352 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 175352 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 175352 T426 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 175352 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 175355 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 175356 T401 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39052",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:39052_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 175357 T401 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 175357 T401 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 175363 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 175469 T425 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 175469 T410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 175469 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 176353 T426 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 176353 T426 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1388763209719/collection1
[junit4] 2> 176354 T426 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 176355 T426 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 176355 T426 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 176358 T426 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1388763209719/collection1/'
[junit4] 2> 176364 T426 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1388763209719/collection1/lib/README' to classloader
[junit4] 2> 176367 T426 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1388763209719/collection1/lib/classes/' to classloader
[junit4] 2> 176506 T426 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 176624 T426 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 176727 T426 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 176749 T426 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 177332 T426 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 177335 T426 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 177338 T426 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 177343 T426 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 177382 T426 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 177383 T426 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1388763209719/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty1/
[junit4] 2> 177383 T426 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31714e34
[junit4] 2> 177384 T426 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty1
[junit4] 2> 177384 T426 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty1/index/
[junit4] 2> 177385 T426 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 177385 T426 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty1/index
[junit4] 2> 177386 T426 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=13, maxMergedSegmentMB=61.82421875, floorSegmentMB=1.9921875, forceMergeDeletesPctAllowed=20.459119156739835, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6258587176718375
[junit4] 2> 177387 T426 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5d79331e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3b278445),segFN=segments_1,generation=1}
[junit4] 2> 177387 T426 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 177394 T426 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 177394 T426 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 177394 T426 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 177395 T426 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 177395 T426 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 177396 T426 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 177396 T426 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 177397 T426 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 177397 T426 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 177398 T426 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 177398 T426 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 177399 T426 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 177399 T426 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 177400 T426 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 177400 T426 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 177401 T426 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 177402 T426 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 177418 T426 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 177423 T426 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 177424 T426 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 177425 T426 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=53.09375, floorSegmentMB=1.7001953125, forceMergeDeletesPctAllowed=9.78988450627622, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21570963271496765
[junit4] 2> 177426 T426 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5d79331e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3b278445),segFN=segments_1,generation=1}
[junit4] 2> 177426 T426 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 177426 T426 oass.SolrIndexSearcher.<init> Opening Searcher@35e7e35b main
[junit4] 2> 177432 T427 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@35e7e35b main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 177436 T426 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 177437 T374 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 177437 T374 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 177437 T430 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39052 collection:collection1 shard:shard1
[junit4] 2> 177440 T430 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 177458 T430 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 177469 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 177469 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 177469 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 177470 T430 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 177470 T430 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C310 name=collection1 org.apache.solr.core.SolrCore@7c5f612a url=http://127.0.0.1:39052/collection1 node=127.0.0.1:39052_ C310_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:39052, core=collection1, node_name=127.0.0.1:39052_}
[junit4] 2> 177470 T430 C310 P39052 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39052/collection1/
[junit4] 2> 177470 T430 C310 P39052 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 177471 T430 C310 P39052 oasc.SyncStrategy.syncToMe http://127.0.0.1:39052/collection1/ has no replicas
[junit4] 2> 177471 T430 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39052/collection1/ shard1
[junit4] 2> 177471 T430 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 177471 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 177477 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 177488 T410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 177488 T425 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 177488 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 177505 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 177510 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 177614 T410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 177614 T425 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 177614 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 177656 T430 oasc.ZkController.register We are http://127.0.0.1:39052/collection1/ and leader is http://127.0.0.1:39052/collection1/
[junit4] 2> 177656 T430 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39052
[junit4] 2> 177656 T430 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 177656 T430 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 177656 T430 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 177658 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 177658 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 177658 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 177659 T430 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 177660 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 177661 T401 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:39052",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:39052_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 177665 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 177724 T374 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 177727 T374 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:39063
[junit4] 2> 177728 T374 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 177728 T374 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 177729 T374 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1388763213319
[junit4] 2> 177729 T374 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1388763213319/'
[junit4] 2> 177767 T374 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1388763213319/solr.xml
[junit4] 2> 177769 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 177769 T425 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 177769 T410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 177858 T374 oasc.CoreContainer.<init> New CoreContainer 940015700
[junit4] 2> 177859 T374 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1388763213319/]
[junit4] 2> 177861 T374 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 177861 T374 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 177862 T374 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 177862 T374 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 177862 T374 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 177863 T374 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 177863 T374 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 177864 T374 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 177864 T374 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 177877 T374 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 177877 T374 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 177878 T374 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 177878 T374 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39030/solr
[junit4] 2> 177878 T374 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 177880 T374 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 177915 T442 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ef63dd5 name:ZooKeeperConnection Watcher:127.0.0.1:39030 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 177916 T374 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 177920 T374 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 177930 T445 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5864d000 name:ZooKeeperConnection Watcher:127.0.0.1:39030/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 177931 T374 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 177937 T374 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 178942 T374 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39063_
[junit4] 2> 178944 T374 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39063_
[junit4] 2> 178948 T410 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 178948 T400 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 178948 T425 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 178948 T445 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 178959 T446 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 178959 T446 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 178961 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 178961 T446 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 178961 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 178962 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 178963 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 178964 T401 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39063",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:39063_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 178965 T401 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 178965 T401 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 178968 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 179072 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 179072 T410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 179072 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 179072 T425 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 179962 T446 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 179962 T446 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1388763213319/collection1
[junit4] 2> 179962 T446 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 179964 T446 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 179964 T446 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 179967 T446 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1388763213319/collection1/'
[junit4] 2> 179971 T446 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1388763213319/collection1/lib/README' to classloader
[junit4] 2> 179972 T446 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1388763213319/collection1/lib/classes/' to classloader
[junit4] 2> 180110 T446 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 180266 T446 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 180368 T446 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 180396 T446 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 181598 T446 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 181606 T446 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 181609 T446 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 181617 T446 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 181686 T446 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 181687 T446 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1388763213319/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2/
[junit4] 2> 181687 T446 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31714e34
[junit4] 2> 181689 T446 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2
[junit4] 2> 181689 T446 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2/index/
[junit4] 2> 181689 T446 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 181690 T446 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2/index
[junit4] 2> 181691 T446 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=13, maxMergedSegmentMB=61.82421875, floorSegmentMB=1.9921875, forceMergeDeletesPctAllowed=20.459119156739835, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6258587176718375
[junit4] 2> 181693 T446 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3c5fb96c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@378b29ff),segFN=segments_1,generation=1}
[junit4] 2> 181693 T446 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 181705 T446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 181706 T446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 181706 T446 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 181707 T446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 181707 T446 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 181707 T446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 181708 T446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 181709 T446 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 181709 T446 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 181710 T446 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 181711 T446 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 181711 T446 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 181712 T446 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 181713 T446 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 181714 T446 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 181715 T446 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 181715 T446 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 181746 T446 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 181754 T446 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 181755 T446 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 181756 T446 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=53.09375, floorSegmentMB=1.7001953125, forceMergeDeletesPctAllowed=9.78988450627622, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21570963271496765
[junit4] 2> 181757 T446 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3c5fb96c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@378b29ff),segFN=segments_1,generation=1}
[junit4] 2> 181757 T446 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 181758 T446 oass.SolrIndexSearcher.<init> Opening Searcher@75d8d503 main
[junit4] 2> 181764 T447 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75d8d503 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 181768 T446 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 181769 T374 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
[junit4] 2> 181770 T374 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 181769 T450 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39063 collection:collection1 shard:shard1
[junit4] 2> 181771 T374 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 181776 T450 oasc.ZkController.register We are http://127.0.0.1:39063/collection1/ and leader is http://127.0.0.1:39052/collection1/
[junit4] 2> 181777 T450 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39063
[junit4] 2> 181777 T450 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 181778 T450 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C311 name=collection1 org.apache.solr.core.SolrCore@15d3fdf4 url=http://127.0.0.1:39063/collection1 node=127.0.0.1:39063_ C311_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:39063, core=collection1, node_name=127.0.0.1:39063_}
[junit4] 2> 181779 T453 C311 P39063 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 181780 T453 C311 P39063 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 181780 T453 C311 P39063 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 181780 T450 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 181780 T453 C311 P39063 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> ASYNC NEW_CORE C312 name=collection1 org.apache.solr.core.SolrCore@6f4ec2e4 url=http://127.0.0.1:39040/collection1 node=127.0.0.1:39040_ C312_STATE=coll:control_collection core:collection1 props:{state=active, base_url=http://127.0.0.1:39040, core=collection1, node_name=127.0.0.1:39040_, leader=true}
[junit4] 2> 181791 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-0 (1456223779729440769)]} 0 4
[junit4] 2> 181791 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-0 (1456223779729440768)]} 0 4
[junit4] 2> 181793 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 181794 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 181794 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 181797 T413 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 181797 T401 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 181799 T401 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "base_url":"http://127.0.0.1:39063",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:39063_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 181824 T400 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 181826 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 181826 T410 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 181826 T425 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 181826 T445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> C311_STATE=coll:collection1 core:collection1 props:{state=recovering, base_url=http://127.0.0.1:39063, core=collection1, node_name=127.0.0.1:39063_}
[junit4] 2> 181935 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-0 (1456223779780820992)]} 0 4
[junit4] 2> 181935 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[2-0 (1456223779780820993)]} 0 4
[junit4] 2> ASYNC NEW_CORE C313 name=collection1 org.apache.solr.core.SolrCore@7c5f612a url=http://127.0.0.1:39052/collection1 node=127.0.0.1:39052_ C313_STATE=coll:collection1 core:collection1 props:{state=active, base_url=http://127.0.0.1:39052, core=collection1, node_name=127.0.0.1:39052_, leader=true}
[junit4] 2> 181937 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-0 (1456223779780820992)]} 0 107
[junit4] 2> 181937 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-0 (1456223779780820993)]} 0 108
[junit4] 2> 181952 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-1 (1456223779901407232)]} 0 1
[junit4] 2> 181953 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-1 (1456223779902455808)]} 0 1
[junit4] 2> 181962 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-1 (1456223779907698689)]} 0 1
[junit4] 2> 181962 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[2-1 (1456223779907698688)]} 0 1
[junit4] 2> 181964 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-1 (1456223779907698689)]} 0 7
[junit4] 2> 181964 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-1 (1456223779907698688)]} 0 7
[junit4] 2> 181975 T374 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 39063
[junit4] 2> 181975 T374 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=940015700
[junit4] 2> 181976 T374 oasc.RecoveryStrategy.close WARN Stopping recovery for zkNodeName=core_node2core=collection1
[junit4] 2> 182053 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&CONTROL=TRUE&wt=javabin} {delete=[1-1 (-1456223780007313408)]} 0 1
[junit4] 2> 182053 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&CONTROL=TRUE&wt=javabin} {delete=[2-1 (-1456223780008361984)]} 0 1
[junit4] 2> 182062 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {delete=[1-1 (-1456223780013604864)]} 0 0
[junit4] 2> 182062 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {delete=[2-1 (-1456223780013604865)]} 0 0
[junit4] 2> 182064 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {delete=[1-1 (-1456223780013604864)]} 0 7
[junit4] 2> 182064 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {delete=[2-1 (-1456223780013604865)]} 0 7
[junit4] 2> 182069 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-2 (1456223780024090624)]} 0 2
[junit4] 2> 182069 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-2 (1456223780024090625)]} 0 2
[junit4] 2> 182078 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[2-2 (1456223780029333504)]} 0 1
[junit4] 2> 182078 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-2 (1456223780029333505)]} 0 1
[junit4] 2> 182080 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-2 (1456223780029333504)]} 0 8
[junit4] 2> 182081 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-2 (1456223780029333505)]} 0 9
[junit4] 2> 182149 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-3 (1456223780107976704)]} 0 2
[junit4] 2> 182149 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-3 (1456223780109025280)]} 0 1
[junit4] 2> 182156 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[2-3 (1456223780113219584)]} 0 1
[junit4] 2> 182157 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-3 (1456223780113219584)]} 0 5
[junit4] 2> 182158 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-3 (1456223780113219585)]} 0 1
[junit4] 2> 182159 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-3 (1456223780113219585)]} 0 6
[junit4] 2> 182174 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-4 (1456223780135239680)]} 0 1
[junit4] 2> 182175 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-4 (1456223780136288256)]} 0 1
[junit4] 2> 182181 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[2-4 (1456223780138385408)]} 0 1
[junit4] 2> 182182 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-4 (1456223780138385408)]} 0 5
[junit4] 2> 182182 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-4 (1456223780139433984)]} 0 1
[junit4] 2> 182183 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-4 (1456223780139433984)]} 0 5
[junit4] 2> 182198 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&CONTROL=TRUE&wt=javabin} {delete=[2-3 (-1456223780159356928)]} 0 1
[junit4] 2> 182199 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&CONTROL=TRUE&wt=javabin} {delete=[1-3 (-1456223780161454080)]} 0 1
[junit4] 2> 182205 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {delete=[2-3 (-1456223780164599808)]} 0 1
[junit4] 2> 182206 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {delete=[2-3 (-1456223780164599808)]} 0 5
[junit4] 2> 182206 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {delete=[1-3 (-1456223780164599809)]} 0 1
[junit4] 2> 182207 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {delete=[1-3 (-1456223780164599809)]} 0 5
[junit4] 2> 182210 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-5 (1456223780171939840)]} 0 1
[junit4] 2> 182211 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-5 (1456223780174036992)]} 0 1
[junit4] 2> 182218 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[2-5 (1456223780176134144)]} 0 1
[junit4] 2> 182219 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-5 (1456223780177182720)]} 0 0
[junit4] 2> 182219 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-5 (1456223780176134144)]} 0 6
[junit4] 2> 182219 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-5 (1456223780177182720)]} 0 5
[junit4] 2> 182261 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-6 (1456223780225417217)]} 0 1
[junit4] 2> 182261 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-6 (1456223780225417216)]} 0 1
[junit4] 2> 182271 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-6 (1456223780231708672)]} 0 1
[junit4] 2> 182271 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[2-6 (1456223780231708673)]} 0 1
[junit4] 2> 182272 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-6 (1456223780231708672)]} 0 7
[junit4] 2> 182273 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-6 (1456223780231708673)]} 0 8
[junit4] 2> 182322 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&CONTROL=TRUE&wt=javabin} {delete=[1-6 (-1456223780290428929)]} 0 0
[junit4] 2> 182322 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&CONTROL=TRUE&wt=javabin} {delete=[2-6 (-1456223780290428928)]} 0 0
[junit4] 2> 182330 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {delete=[2-6 (-1456223780295671809)]} 0 0
[junit4] 2> 182330 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {delete=[1-6 (-1456223780295671808)]} 0 0
[junit4] 2> 182331 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {delete=[2-6 (-1456223780295671809)]} 0 5
[junit4] 2> 182332 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {delete=[1-6 (-1456223780295671808)]} 0 6
[junit4] 2> 182336 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-7 (1456223780305108992)]} 0 1
[junit4] 2> 182336 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-7 (1456223780305108993)]} 0 1
[junit4] 2> 182344 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-7 (1456223780310351873)]} 0 0
[junit4] 2> 182344 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[2-7 (1456223780310351872)]} 0 0
[junit4] 2> 182345 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-7 (1456223780310351873)]} 0 5
[junit4] 2> 182346 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-7 (1456223780310351872)]} 0 6
[junit4] 2> 182359 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-8 (1456223780329226240)]} 0 1
[junit4] 2> 182360 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-8 (1456223780330274816)]} 0 1
[junit4] 2> 182366 T433 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-8 (1456223780332371968)]} 0 1
[junit4] 2> 182367 T414 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-8 (1456223780332371968)]} 0 5
[junit4] 2> 182368 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[2-8 (1456223780334469120)]} 0 2
[junit4] 2> 182369 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-8 (1456223780334469120)]} 0 6
[junit4] 2> 182382 T388 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-9 (1456223780353343488)]} 0 1
[junit4] 2> 182383 T389 C312 P39040 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2-9 (1456223780354392064)]} 0 1
[junit4] 2> 182389 T434 C311 P39063 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:39052/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1-9 (1456223780356489216)]} 0 1
[junit4] 2> 182390 T415 C313 P39052 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1-9 (1456223780356489216)]} 0 5
[junit4] 2> 182391 T433 C311 P39063 oas
[...truncated too long message...]
eJ4.deleteCore ###deleteCore
[junit4] 2> 204746 T400 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 204746 T400 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 204838 T425 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 204838 T425 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 205861 T475 C327 P39063 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 205861 T475 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 205862 T475 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1d66679d
[junit4] 2> 205872 T475 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=3,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=77,cumulative_deletesById=39,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=1}
[junit4] 2> 205873 T475 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 205873 T475 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 205873 T475 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 205874 T475 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 205875 T475 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 205875 T475 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2/index.20140103213346872 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2/index.20140103213346872;done=false>>]
[junit4] 2> 205875 T475 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2/index.20140103213346872
[junit4] 2> 35984 T373 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> 205929 T475 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2;done=false>>]
[junit4] 2> 205929 T475 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1388763205831/jetty2
[junit4] 2> 205930 T475 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 205930 T475 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: test params are: codec=Lucene42, sim=DefaultSimilarity, locale=ar_JO, timezone=Asia/Qyzylorda
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=1,free=48951864,total=194707456
[junit4] 2> NOTE: All tests run in this JVM: [TestJoin, TestDocSet, TestStressVersions, ShowFileRequestHandlerTest, DeleteReplicaTest, TestSolrQueryParser, ExpressionTest, CircularListTest, QueryParsingTest, OverseerCollectionProcessorTest, WordBreakSolrSpellCheckerTest, TestReversedWildcardFilterFactory, TestFaceting, PeerSyncTest, HdfsUnloadDistributedZkTest, ResourceLoaderTest, TestFieldCollectionResource, SolrIndexSplitterTest, ScriptEngineTest, RecoveryZkTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RecoveryZkTest -Dtests.seed=DAC754BB48CAF363 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_JO -Dtests.timezone=Asia/Qyzylorda -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | RecoveryZkTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=476, name=coreLoadExecutor-217-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
[junit4] > at __randomizedtesting.SeedInfo.seed([DAC754BB48CAF363]:0)
[junit4] > at java.lang.Thread.getStackTrace(Thread.java:1568)
[junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
[junit4] > at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:551)
[junit4] > at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1035)
[junit4] > at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:933)
[junit4] > at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:925)
[junit4] > at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:350)
[junit4] > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:254)
[junit4] > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:468)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:759)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:628)
[junit4] > at org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:223)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:592)
[junit4] > at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:258)
[junit4] > at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:250)
[junit4] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4] > Caused by: java.lang.AssertionError: Directory not closed: BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@2e1c837f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51fd1e7f)
[junit4] > at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
[junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
[junit4] > ... 1 more
[junit4] Completed on J0 in 36.29s, 1 test, 1 error <<< FAILURES!
[...truncated 1060 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:460: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:433: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:491: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1304: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:937: There were test failures: 358 suites, 1577 tests, 1 suite-level error, 33 ignored (4 assumptions)
Total time: 134 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 486 - Still Failing
Posted by Mark Miller <ma...@gmail.com>.
Just tried again on this one.
- Mark
On Jan 5, 2014, at 1:08 PM, Apache Jenkins Server <je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/486/
>
> 2 tests failed.
> FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardSplitTest
>
> Error Message:
> 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest: 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at java.util.TimerThread.mainLoop(Timer.java:526) at java.util.TimerThread.run(Timer.java:505)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest:
> 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:503)
> at java.util.TimerThread.mainLoop(Timer.java:526)
> at java.util.TimerThread.run(Timer.java:505)
> at __randomizedtesting.SeedInfo.seed([2B6E400BDCDB9BD9]:0)
>
>
> FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardSplitTest
>
> Error Message:
> There are still zombie threads that couldn't be terminated: 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at java.util.TimerThread.mainLoop(Timer.java:526) at java.util.TimerThread.run(Timer.java:505)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
> 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:503)
> at java.util.TimerThread.mainLoop(Timer.java:526)
> at java.util.TimerThread.run(Timer.java:505)
> at __randomizedtesting.SeedInfo.seed([2B6E400BDCDB9BD9]:0)
>
>
>
>
> Build Log:
> [...truncated 11143 lines...]
> [junit4] Suite: org.apache.solr.cloud.ShardSplitTest
> [junit4] 2> 2832647 T7686 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
> [junit4] 2> 2832653 T7686 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./solrtest-ShardSplitTest-1388944892812
> [junit4] 2> 2832654 T7686 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4] 2> 2832655 T7687 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4] 2> 2832756 T7686 oasc.ZkTestServer.run start zk server on port:60363
> [junit4] 2> 2832757 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2832761 T7694 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b00ae47 name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2832761 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2832762 T7686 oascc.SolrZkClient.makePath makePath: /solr
> [junit4] 2> 2832789 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2832790 T7697 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f2a7f37 name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2832791 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2832791 T7686 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4] 2> 2832795 T7686 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4] 2> 2832804 T7686 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4] 2> 2832806 T7686 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4] 2> 2832808 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
> [junit4] 2> 2832809 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4] 2> 2832814 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
> [junit4] 2> 2832815 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4] 2> 2832918 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 2832919 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 2832929 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4] 2> 2832930 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4] 2> 2832933 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4] 2> 2832933 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4] 2> 2832936 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4] 2> 2832937 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4] 2> 2832940 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
> [junit4] 2> 2832940 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
> [junit4] 2> 2832943 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4] 2> 2832944 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4] 2> 2832946 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 2832947 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 2832950 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4] 2> 2832950 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4] 2> 2832953 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4] 2> 2832954 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4] 2> 2833239 T7686 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 2833244 T7686 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44202
> [junit4] 2> 2833245 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 2833245 T7686 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 2833246 T7686 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121
> [junit4] 2> 2833246 T7686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/'
> [junit4] 2> 2833285 T7686 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/solr.xml
> [junit4] 2> 2833378 T7686 oasc.CoreContainer.<init> New CoreContainer 542755691
> [junit4] 2> 2833378 T7686 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/]
> [junit4] 2> 2833380 T7686 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 2833381 T7686 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4] 2> 2833381 T7686 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 2833381 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 2833382 T7686 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 2833382 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 2833383 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 2833383 T7686 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 2833383 T7686 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 2833395 T7686 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 2833396 T7686 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 2833396 T7686 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 2833397 T7686 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60363/solr
> [junit4] 2> 2833397 T7686 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 2833398 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2833401 T7709 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c521c2e name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2833402 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2833405 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2833413 T7712 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e8c6f27 name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2833413 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2833416 T7686 oascc.SolrZkClient.makePath makePath: /overseer/queue
> [junit4] 2> 2833427 T7686 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
> [junit4] 2> 2833429 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4] 2> 2833432 T7686 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44202_
> [junit4] 2> 2833439 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44202_
> [junit4] 2> 2833442 T7686 oascc.SolrZkClient.makePath makePath: /overseer_elect
> [junit4] 2> 2833445 T7686 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4] 2> 2833460 T7686 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4] 2> 2833462 T7686 oasc.Overseer.start Overseer (id=91025892495851523-127.0.0.1:44202_-n_0000000000) starting
> [junit4] 2> 2833474 T7686 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
> [junit4] 2> 2833485 T7714 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4] 2> 2833487 T7686 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4] 2> 2833489 T7686 oascc.SolrZkClient.makePath makePath: /aliases.json
> [junit4] 2> 2833491 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 2833493 T7713 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4] 2> 2833499 T7715 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4] 2> 2833499 T7715 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 2833501 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2833501 T7715 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 2833502 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2833502 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:44202",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:44202_",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"control_collection",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 2833503 T7713 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
> [junit4] 2> 2833503 T7713 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4] 2> 2833505 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2833514 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 2834502 T7715 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 2834502 T7715 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1
> [junit4] 2> 2834502 T7715 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4] 2> 2834503 T7715 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 2834503 T7715 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
> [junit4] 2> 2834504 T7715 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1/'
> [junit4] 2> 2834506 T7715 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1/lib/README' to classloader
> [junit4] 2> 2834507 T7715 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1/lib/classes/' to classloader
> [junit4] 2> 2834569 T7715 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4] 2> 2834630 T7715 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 2834732 T7715 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 2834742 T7715 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 2835300 T7715 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 2835305 T7715 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 2835307 T7715 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 2835312 T7715 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 2835332 T7715 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 2835332 T7715 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data/
> [junit4] 2> 2835332 T7715 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c2bae18
> [junit4] 2> 2835333 T7715 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data
> [junit4] 2> 2835333 T7715 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data/index/
> [junit4] 2> 2835334 T7715 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data/index' doesn't exist. Creating new index...
> [junit4] 2> 2835334 T7715 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data/index
> [junit4] 2> 2835335 T7715 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=51.1748046875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=20.22639910826647, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.361328125, noCFSRatio=1.0
> [junit4] 2> 2835336 T7715 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@4a1266b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5c49f02f; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
> [junit4] 2> 2835336 T7715 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 2835339 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 2835339 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 2835340 T7715 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 2835340 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 2835340 T7715 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4] 2> 2835340 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 2835341 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 2835341 T7715 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 2835342 T7715 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 2835342 T7715 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4] 2> 2835343 T7715 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4] 2> 2835343 T7715 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 2835343 T7715 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4] 2> 2835344 T7715 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4] 2> 2835344 T7715 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 2835345 T7715 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
> [junit4] 2> 2835345 T7715 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
> [junit4] 2> 2835361 T7715 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4] 2> 2835378 T7715 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4] 2> 2835378 T7715 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4] 2> 2835379 T7715 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5898393679458741]
> [junit4] 2> 2835380 T7715 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@4a1266b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5c49f02f; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
> [junit4] 2> 2835380 T7715 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 2835380 T7715 oass.SolrIndexSearcher.<init> Opening Searcher@5278ff5e main
> [junit4] 2> 2835383 T7716 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5278ff5e main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 2835385 T7715 oasc.CoreContainer.registerCore registering core: collection1
> [junit4] 2> 2835386 T7686 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
> [junit4] 2> 2835386 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4] 2> 2835386 T7719 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44202 collection:control_collection shard:shard1
> [junit4] 2> 2835388 T7719 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4] 2> 2835388 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2835390 T7722 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11ce6ebf name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2835390 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2835392 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 2835405 T7686 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
> [junit4] 2> 2835406 T7719 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4] 2> 2835408 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2835409 T7719 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4] 2> 2835409 T7719 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4] 2> ASYNC NEW_CORE C2589 name=collection1 org.apache.solr.core.SolrCore@15dee390 url=http://127.0.0.1:44202/collection1 node=127.0.0.1:44202_ C2589_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:44202, core=collection1, node_name=127.0.0.1:44202_}
> [junit4] 2> 2835409 T7719 C2589 P44202 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44202/collection1/
> [junit4] 2> 2835410 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2835410 T7719 C2589 P44202 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4] 2> 2835410 T7719 C2589 P44202 oasc.SyncStrategy.syncToMe http://127.0.0.1:44202/collection1/ has no replicas
> [junit4] 2> 2835410 T7719 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44202/collection1/ shard1
> [junit4] 2> 2835411 T7719 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4] 2> 2835413 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2835413 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 2835413 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 2835418 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2835421 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2835523 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 2835523 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 2835571 T7719 oasc.ZkController.register We are http://127.0.0.1:44202/collection1/ and leader is http://127.0.0.1:44202/collection1/
> [junit4] 2> 2835571 T7719 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44202
> [junit4] 2> 2835571 T7719 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4] 2> 2835572 T7719 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4] 2> 2835572 T7719 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 2835613 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2835613 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2835614 T7719 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2835614 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2835615 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2835615 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"http://127.0.0.1:44202",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:44202_",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"control_collection",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 2835682 T7686 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 2835685 T7686 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56357
> [junit4] 2> 2835686 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 2835687 T7686 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 2835687 T7686 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564
> [junit4] 2> 2835687 T7686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/'
> [junit4] 2> 2835704 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2835726 T7686 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/solr.xml
> [junit4] 2> 2835807 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 2835807 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 2835818 T7686 oasc.CoreContainer.<init> New CoreContainer 1658576934
> [junit4] 2> 2835819 T7686 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/]
> [junit4] 2> 2835820 T7686 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 2835821 T7686 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4] 2> 2835821 T7686 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 2835822 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 2835822 T7686 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 2835822 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 2835823 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 2835823 T7686 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 2835824 T7686 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 2835833 T7686 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 2835833 T7686 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 2835834 T7686 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 2835834 T7686 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60363/solr
> [junit4] 2> 2835835 T7686 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 2835836 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2835875 T7734 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e8b90eb name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2835877 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2835900 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2835902 T7737 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ceeeab4 name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2835902 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2835948 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 2836951 T7686 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56357_
> [junit4] 2> 2836953 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56357_
> [junit4] 2> 2836957 T7722 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4] 2> 2836957 T7737 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4] 2> 2836957 T7712 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4] 2> 2836979 T7738 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4] 2> 2836979 T7738 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 2836981 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2836981 T7738 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 2836981 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2836981 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2836982 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2836983 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:56357",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:56357_",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 2836983 T7713 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
> [junit4] 2> 2836984 T7713 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4] 2> 2836987 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2837090 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2837090 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2837090 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2837982 T7738 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 2837982 T7738 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1
> [junit4] 2> 2837982 T7738 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4] 2> 2837983 T7738 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 2837983 T7738 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 2837984 T7738 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1/'
> [junit4] 2> 2837985 T7738 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1/lib/README' to classloader
> [junit4] 2> 2837986 T7738 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1/lib/classes/' to classloader
> [junit4] 2> 2838048 T7738 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4] 2> 2838109 T7738 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 2838210 T7738 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 2838219 T7738 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 2838764 T7738 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 2838767 T7738 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 2838769 T7738 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 2838772 T7738 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 2838790 T7738 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 2838791 T7738 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1/
> [junit4] 2> 2838791 T7738 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c2bae18
> [junit4] 2> 2838791 T7738 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1
> [junit4] 2> 2838792 T7738 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1/index/
> [junit4] 2> 2838792 T7738 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1/index' doesn't exist. Creating new index...
> [junit4] 2> 2838793 T7738 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1/index
> [junit4] 2> 2838794 T7738 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=51.1748046875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=20.22639910826647, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.361328125, noCFSRatio=1.0
> [junit4] 2> 2838794 T7738 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@381705c5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7d67c554; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
> [junit4] 2> 2838795 T7738 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 2838797 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 2838797 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 2838798 T7738 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 2838798 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 2838798 T7738 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4] 2> 2838798 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 2838799 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 2838799 T7738 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 2838799 T7738 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 2838800 T7738 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4] 2> 2838800 T7738 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4] 2> 2838800 T7738 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 2838800 T7738 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4] 2> 2838801 T7738 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4] 2> 2838801 T7738 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 2838801 T7738 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
> [junit4] 2> 2838802 T7738 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
> [junit4] 2> 2838812 T7738 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4] 2> 2838816 T7738 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4] 2> 2838816 T7738 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4] 2> 2838817 T7738 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5898393679458741]
> [junit4] 2> 2838817 T7738 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@381705c5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7d67c554; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
> [junit4] 2> 2838817 T7738 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 2838818 T7738 oass.SolrIndexSearcher.<init> Opening Searcher@5a4b96db main
> [junit4] 2> 2838820 T7739 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a4b96db main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 2838822 T7738 oasc.CoreContainer.registerCore registering core: collection1
> [junit4] 2> 2838823 T7686 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
> [junit4] 2> 2838824 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4] 2> 2838823 T7742 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56357 collection:collection1 shard:shard1
> [junit4] 2> 2838825 T7742 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4] 2> 2838844 T7742 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4] 2> 2838855 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2838855 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2838855 T7742 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4] 2> 2838855 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2838856 T7742 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4] 2> ASYNC NEW_CORE C2590 name=collection1 org.apache.solr.core.SolrCore@6f0b6c21 url=http://127.0.0.1:56357/collection1 node=127.0.0.1:56357_ C2590_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:56357, core=collection1, node_name=127.0.0.1:56357_}
> [junit4] 2> 2838856 T7742 C2590 P56357 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56357/collection1/
> [junit4] 2> 2838856 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2838856 T7742 C2590 P56357 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4] 2> 2838857 T7742 C2590 P56357 oasc.SyncStrategy.syncToMe http://127.0.0.1:56357/collection1/ has no replicas
> [junit4] 2> 2838857 T7742 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56357/collection1/ shard1
> [junit4] 2> 2838857 T7742 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4] 2> 2838859 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2838860 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2838860 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2838860 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2838863 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2838864 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2838866 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2838969 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2838969 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2838969 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2839016 T7742 oasc.ZkController.register We are http://127.0.0.1:56357/collection1/ and leader is http://127.0.0.1:56357/collection1/
> [junit4] 2> 2839016 T7742 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56357
> [junit4] 2> 2839016 T7742 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4] 2> 2839016 T7742 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4] 2> 2839016 T7742 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 2839017 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2839018 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2839018 T7742 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2839018 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2839019 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2839020 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"http://127.0.0.1:56357",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:56357_",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 2839029 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2839096 T7686 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 2839099 T7686 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38382
> [junit4] 2> 2839100 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 2839100 T7686 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 2839101 T7686 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982
> [junit4] 2> 2839101 T7686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/'
> [junit4] 2> 2839131 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2839131 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2839131 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 2839141 T7686 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/solr.xml
> [junit4] 2> 2839233 T7686 oasc.CoreContainer.<init> New CoreContainer 2064723613
> [junit4] 2> 2839234 T7686 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/]
> [junit4] 2> 2839236 T7686 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 2839236 T7686 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4] 2> 2839237 T7686 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 2839237 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 2839238 T7686 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 2839238 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 2839238 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 2839239 T7686 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 2839239 T7686 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 2839251 T7686 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 2839251 T7686 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 2839252 T7686 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 2839252 T7686 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60363/solr
> [junit4] 2> 2839253 T7686 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 2839254 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2839256 T7754 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@172a9306 name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2839257 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2839261 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2839267 T7757 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a88a8f6 name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2839268 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2839272 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 2840275 T7686 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38382_
> [junit4] 2> 2840277 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38382_
> [junit4] 2> 2840281 T7722 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4] 2> 2840282 T7757 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4] 2> 2840281 T7737 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4] 2> 2840281 T7712 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4] 2> 2840298 T7758 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4] 2> 2840298 T7758 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 2840299 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2840299 T7758 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 2840300 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2840300 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2840301 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2840301 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:38382",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:38382_",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 2840302 T7713 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4] 2> 2840302 T7713 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4] 2> 2840305 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2840408 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2840408 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2840408 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2840408 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2841301 T7758 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 2841301 T7758 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1
> [junit4] 2> 2841301 T7758 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4] 2> 2841302 T7758 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 2841303 T7758 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 2841303 T7758 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1/'
> [junit4] 2> 2841305 T7758 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1/lib/README' to classloader
> [junit4] 2> 2841306 T7758 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1/lib/classes/' to classloader
> [junit4] 2> 2841370 T7758 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4] 2> 2841442 T7758 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 2841543 T7758 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 2841553 T7758 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 2841996 T7758 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 2841999 T7758 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 2842000 T7758 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 2842004 T7758 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 2842018 T7758 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 2842018 T7758 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2/
> [junit4] 2> 2842018 T7758 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c2bae18
> [junit4] 2> 2842019 T7758 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2
> [junit4] 2> 2842019 T7758 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2/index/
> [junit4] 2> 2842019 T7758 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2/index' doesn't exist. Creating new index...
> [junit4] 2> 2842020 T7758 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2/index
> [junit4] 2> 2842021 T7758 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=51.1748046875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=20.22639910826647, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.361328125, noCFSRatio=1.0
> [junit4] 2> 2842021 T7758 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@4fdd011 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5289967c; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
> [junit4] 2> 2842022 T7758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 2842024 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 2842026 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 2842026 T7758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 2842026 T7758 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 2842027 T7758 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4] 2> 2842027 T7758 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4] 2> 2842027 T7758 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 2842027 T7758 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4] 2> 2842028 T7758 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4] 2> 2842028 T7758 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 2842029 T7758 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
> [junit4] 2> 2842029 T7758 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
> [junit4] 2> 2842040 T7758 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4] 2> 2842043 T7758 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4] 2> 2842044 T7758 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4] 2> 2842044 T7758 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5898393679458741]
> [junit4] 2> 2842044 T7758 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@4fdd011 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5289967c; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
> [junit4] 2> 2842045 T7758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 2842045 T7758 oass.SolrIndexSearcher.<init> Opening Searcher@2f5b1603 main
> [junit4] 2> 2842047 T7759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f5b1603 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 2842049 T7758 oasc.CoreContainer.registerCore registering core: collection1
> [junit4] 2> 2842049 T7686 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
> [junit4] 2> 2842050 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4] 2> 2842049 T7762 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38382 collection:collection1 shard:shard2
> [junit4] 2> 2842051 T7762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4] 2> 2842067 T7762 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
> [junit4] 2> 2842069 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2842069 T7762 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4] 2> 2842069 T7762 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4] 2> 2842069 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> ASYNC NEW_CORE C2591 name=collection1 org.apache.solr.core.SolrCore@3327f008 url=http://127.0.0.1:38382/collection1 node=127.0.0.1:38382_ C2591_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:38382, core=collection1, node_name=127.0.0.1:38382_}
> [junit4] 2> 2842069 T7762 C2591 P38382 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38382/collection1/
> [junit4] 2> 2842070 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2842070 T7762 C2591 P38382 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4] 2> 2842070 T7762 C2591 P38382 oasc.SyncStrategy.syncToMe http://127.0.0.1:38382/collection1/ has no replicas
> [junit4] 2> 2842070 T7762 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38382/collection1/ shard2
> [junit4] 2> 2842070 T7762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4] 2> 2842071 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2842074 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2842075 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842075 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842075 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842075 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842077 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2842080 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2842183 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842183 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842183 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842183 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842228 T7762 oasc.ZkController.register We are http://127.0.0.1:38382/collection1/ and leader is http://127.0.0.1:38382/collection1/
> [junit4] 2> 2842228 T7762 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38382
> [junit4] 2> 2842228 T7762 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4] 2> 2842228 T7762 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4] 2> 2842228 T7762 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 2842230 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2842230 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2842230 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2842230 T7762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2842231 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2842231 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"http://127.0.0.1:38382",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:38382_",
> [junit4] 2> "shard":"shard2",
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":"core_node2"}
> [junit4] 2> 2842232 T7686 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 2842234 T7686 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45611
> [junit4] 2> 2842234 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2842235 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 2842235 T7686 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 2842235 T7686 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208
> [junit4] 2> 2842236 T7686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/'
> [junit4] 2> 2842262 T7686 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/solr.xml
> [junit4] 2> 2842323 T7686 oasc.CoreContainer.<init> New CoreContainer 1245378889
> [junit4] 2> 2842323 T7686 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/]
> [junit4] 2> 2842324 T7686 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 2842325 T7686 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4] 2> 2842325 T7686 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 2842325 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 2842326 T7686 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 2842326 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 2842326 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 2842326 T7686 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 2842327 T7686 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 2842334 T7686 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 2842335 T7686 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 2842335 T7686 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 2842335 T7686 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60363/solr
> [junit4] 2> 2842336 T7686 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 2842336 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2842338 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842338 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842338 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842338 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 2842340 T7774 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56c01a65 name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2842340 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2842351 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 2842353 T7777 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23b0b33d name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 2842354 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 2842363 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 2843366 T7686 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45611_
> [junit4] 2> 2843394 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45611_
> [junit4] 2> 2843397 T7777 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 2843398 T7737 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 2843398 T7712 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 2843398 T7757 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 2843397 T7722 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 2843415 T7778 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4] 2> 2843415 T7778 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 2843416 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2843416 T7778 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 2843417 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2843417 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2843418 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 2843418 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"http://127.0.0.1:45611",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:45611_",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"2",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 2843418 T7713 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4] 2> 2843419 T7713 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4] 2> 2843422 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 2843524 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 2843524 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 2843524 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 2843524 T7777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 2843524 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 2844418 T7778 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 2844418 T7778 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1
> [junit4] 2> 2844418 T7778 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4] 2> 2844419 T7778 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 2844419 T7778 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 2844420 T7778 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1/'
> [junit4] 2> 2844421 T7778 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1/lib/classes/' to classloader
> [junit4] 2> 2844422 T7778 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1/lib/README' to classloader
> [junit4] 2> 2844466 T7778 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4] 2> 2844508 T7778 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 2844610 T7778 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 2844617 T7778 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 2845020 T7778 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 2845023 T7778 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 2845025 T7778 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 2845028 T7778 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 2845043 T7778 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 2845043 T7778 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3/
> [junit4] 2> 2845043 T7778 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c2bae18
> [junit4] 2> 2845044 T7778 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3
> [junit4] 2> 2845044 T7778 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3/index/
> [junit4] 2> 2845044 T7778 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3/index' doesn't exist. Creating new index...
> [junit4] 2> 2845046 T7778 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3/index
> [junit4] 2> 2845046 T7778 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=51.1748046875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=20.22639910826647, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.361328125, noCFSRatio=1.0
> [junit4] 2> 2845047 T7778 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@69634371 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e64b14d; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
> [junit4] 2> 2845047 T7778 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 2845050 T7778 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 2845050 T7778 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 2845050 T7778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 2845051 T7778 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 2845051 T7778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_s
>
> [...truncated too long message...]
>
> a:519)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> [junit4] 2>
> [junit4] 2> 3169708 T7798 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
> [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
> [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> [junit4] 2>
> [junit4] 2> 337466 T7685 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
> [junit4] 2> 357525 T7685 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest:
> [junit4] 2> 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
> [junit4] 2> at java.lang.Object.wait(Native Method)
> [junit4] 2> at java.lang.Object.wait(Object.java:503)
> [junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:526)
> [junit4] 2> at java.util.TimerThread.run(Timer.java:505)
> [junit4] 2> 357526 T7685 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
> [junit4] 2> 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
> [junit4] 2> 360533 T7685 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
> [junit4] 2> 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
> [junit4] 2> at java.lang.Object.wait(Native Method)
> [junit4] 2> at java.lang.Object.wait(Object.java:503)
> [junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:526)
> [junit4] 2> at java.util.TimerThread.run(Timer.java:505)
> [junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=pl_PL, timezone=Africa/Sao_Tome
> [junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=2,free=279178864,total=494141440
> [junit4] 2> NOTE: All tests run in this JVM: [TestUniqueKeyFieldResource, RAMDirectoryFactoryTest, DistributedSuggestComponentTest, TestComponentsName, BasicZkTest, ShardRoutingCustomTest, CollectionsAPIDistributedZkTest, SpellingQueryConverterTest, DirectSolrConnectionTest, TestNumberUtils, SolrCmdDistributorTest, TestAnalyzedSuggestions, TestAddFieldRealTimeGet, TestSolrQueryParserDefaultOperatorResource, HighlighterTest, ChangedSchemaMergeTest, TestUpdate, RequiredFieldsTest, BlockDirectoryTest, SimplePostToolTest, TestRangeQuery, TestSolrDeletionPolicy2, AlternateDirectoryTest, TestIndexSearcher, IndexBasedSpellCheckerTest, AliasIntegrationTest, TestBinaryResponseWriter, AddBlockUpdateTest, TestClassNameShortening, TestFastLRUCache, DateFieldTest, TestStressReorder, OutputWriterTest, TestFoldingMultitermQuery, MinimalSchemaTest, TestCoreDiscovery, TestHashPartitioner, TestFieldTypeCollectionResource, TestOmitPositions, TestSolrQueryParserResource, ZkSolrClientTest, TestRTGBase, RequestHandlersTest, TestNRTOpen, StatsComponentTest, TimeZoneUtilsTest, BlockCacheTest, ConnectionManagerTest, TestAtomicUpdateErrorCases, TestSolrXmlPersistence, TestArbitraryIndexDir, StressHdfsTest, TestSweetSpotSimilarityFactory, TestBinaryField, TestStressRecovery, SolrPluginUtilsTest, TestZkChroot, SpellCheckCollatorTest, TestCSVLoader, DebugComponentTest, TestSolr4Spatial, TestDFRSimilarityFactory, XsltUpdateRequestHandlerTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestIBSimilarityFactory, OpenExchangeRatesOrgProviderTest, ChaosMonkeySafeLeaderTest, TestIndexingPerformance, TestCodecSupport, FieldAnalysisRequestHandlerTest, UniqFieldsUpdateProcessorFactoryTest, DocValuesMultiTest, TestSolrJ, TestQuerySenderListener, DistributedQueryComponentCustomSortTest, TestRandomMergePolicy, FieldFacetExtrasTest, DefaultValueUpdateProcessorTest, TestPhraseSuggestions, IndexSchemaRuntimeFieldTest, CachingDirectoryFactoryTest, TestDistribDocBasedVersion, StandardRequestHandlerTest, AbstractAnalyticsStatsTest, TestFastWriter, QueryFacetTest, TestFieldResource, BadIndexSchemaTest, BasicDistributedZk2Test, SyncSliceTest, LeaderElectionIntegrationTest, ClusterStateUpdateTest, TestRandomFaceting, TestDistributedGrouping, TestFaceting, TestRecovery, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestJoin, TestMultiCoreConfBootstrap, TestReload, TestCoreContainer, TestGroupingSearch, DirectUpdateHandlerTest, OverseerCollectionProcessorTest, CurrencyFieldOpenExchangeTest, SolrIndexSplitterTest, AnalysisAfterCoreReloadTest, SolrRequestParserTest, TestStressLucene, PolyFieldTest, SolrCoreCheckLockOnStartupTest, TestWordDelimiterFilterFactory, SortByFunctionTest, TestSolrDeletionPolicy1, LukeRequestHandlerTest, DirectSolrSpellCheckerTest, TestQueryTypes, TermsComponentTest, TestValueSourceCache, FastVectorHighlighterTest, TestJmxIntegration, TestCSVResponseWriter, JsonLoaderTest, TestPHPSerializedResponseWriter, SearchHandlerTest, HighlighterConfigTest, UpdateParamsTest, TestQuerySenderNoQuery, ResponseLogComponentTest, BadComponentTest, TestSearchPerf, TestFuzzyAnalyzedSuggestions, TestSolrCoreProperties, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPerFieldSimilarity, TestDefaultSimilarityFactory, TestFastOutputStream, URLClassifyProcessorTest, PreAnalyzedFieldTest, PrimUtilsTest, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SystemInfoHandlerTest, FileUtilsTest, SolrTestCaseJ4Test, TestDistributedMissingSort, ExpressionTest, AbstractAnalyticsFacetTest, FieldFacetTest, FunctionTest, AssignTest, DeleteReplicaTest, DeleteShardTest, MigrateRouteKeyTest, ShardSplitTest]
> [junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ShardSplitTest -Dtests.seed=2B6E400BDCDB9BD9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=pl_PL -Dtests.timezone=Africa/Sao_Tome -Dtests.file.encoding=UTF-8
> [junit4] ERROR 0.00s J1 | ShardSplitTest (suite) <<<
> [junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest:
> [junit4] > 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
> [junit4] > at java.lang.Object.wait(Native Method)
> [junit4] > at java.lang.Object.wait(Object.java:503)
> [junit4] > at java.util.TimerThread.mainLoop(Timer.java:526)
> [junit4] > at java.util.TimerThread.run(Timer.java:505)
> [junit4] > at __randomizedtesting.SeedInfo.seed([2B6E400BDCDB9BD9]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
> [junit4] > 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
> [junit4] > at java.lang.Object.wait(Native Method)
> [junit4] > at java.lang.Object.wait(Object.java:503)
> [junit4] > at java.util.TimerThread.mainLoop(Timer.java:526)
> [junit4] > at java.util.TimerThread.run(Timer.java:505)
> [junit4] > at __randomizedtesting.SeedInfo.seed([2B6E400BDCDB9BD9]:0)
> [junit4] Completed on J1 in 360.59s, 1 test, 2 errors <<< FAILURES!
>
> [...truncated 13 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:460: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:433: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:39: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:491: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1304: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:937: There were test failures: 358 suites, 1577 tests, 2 suite-level errors, 33 ignored (4 assumptions)
>
> Total time: 116 minutes 37 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 486 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/486/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardSplitTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest: 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at java.util.TimerThread.mainLoop(Timer.java:526) at java.util.TimerThread.run(Timer.java:505)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest:
1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at java.util.TimerThread.mainLoop(Timer.java:526)
at java.util.TimerThread.run(Timer.java:505)
at __randomizedtesting.SeedInfo.seed([2B6E400BDCDB9BD9]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardSplitTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at java.util.TimerThread.mainLoop(Timer.java:526) at java.util.TimerThread.run(Timer.java:505)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at java.util.TimerThread.mainLoop(Timer.java:526)
at java.util.TimerThread.run(Timer.java:505)
at __randomizedtesting.SeedInfo.seed([2B6E400BDCDB9BD9]:0)
Build Log:
[...truncated 11143 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> 2832647 T7686 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 2832653 T7686 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./solrtest-ShardSplitTest-1388944892812
[junit4] 2> 2832654 T7686 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 2832655 T7687 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 2832756 T7686 oasc.ZkTestServer.run start zk server on port:60363
[junit4] 2> 2832757 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2832761 T7694 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b00ae47 name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2832761 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2832762 T7686 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2832789 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2832790 T7697 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f2a7f37 name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2832791 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2832791 T7686 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 2832795 T7686 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 2832804 T7686 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 2832806 T7686 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 2832808 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2832809 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 2832814 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 2832815 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 2832918 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2832919 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2832929 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2832930 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 2832933 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2832933 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 2832936 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2832937 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 2832940 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2832940 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 2832943 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2832944 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 2832946 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2832947 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2832950 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2832950 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 2832953 T7686 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2832954 T7686 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 2833239 T7686 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2833244 T7686 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44202
[junit4] 2> 2833245 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2833245 T7686 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2833246 T7686 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121
[junit4] 2> 2833246 T7686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/'
[junit4] 2> 2833285 T7686 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/solr.xml
[junit4] 2> 2833378 T7686 oasc.CoreContainer.<init> New CoreContainer 542755691
[junit4] 2> 2833378 T7686 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/]
[junit4] 2> 2833380 T7686 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2833381 T7686 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2833381 T7686 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2833381 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2833382 T7686 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2833382 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2833383 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2833383 T7686 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2833383 T7686 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2833395 T7686 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2833396 T7686 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2833396 T7686 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2833397 T7686 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60363/solr
[junit4] 2> 2833397 T7686 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2833398 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2833401 T7709 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c521c2e name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2833402 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2833405 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2833413 T7712 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e8c6f27 name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2833413 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2833416 T7686 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2833427 T7686 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 2833429 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 2833432 T7686 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44202_
[junit4] 2> 2833439 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44202_
[junit4] 2> 2833442 T7686 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 2833445 T7686 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 2833460 T7686 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2833462 T7686 oasc.Overseer.start Overseer (id=91025892495851523-127.0.0.1:44202_-n_0000000000) starting
[junit4] 2> 2833474 T7686 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 2833485 T7714 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2833487 T7686 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 2833489 T7686 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 2833491 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2833493 T7713 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2833499 T7715 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 2833499 T7715 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2833501 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2833501 T7715 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2833502 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2833502 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44202",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:44202_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2833503 T7713 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 2833503 T7713 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2833505 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2833514 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2834502 T7715 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2834502 T7715 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1
[junit4] 2> 2834502 T7715 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 2834503 T7715 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2834503 T7715 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 2834504 T7715 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1/'
[junit4] 2> 2834506 T7715 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1/lib/README' to classloader
[junit4] 2> 2834507 T7715 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1/lib/classes/' to classloader
[junit4] 2> 2834569 T7715 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2834630 T7715 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2834732 T7715 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2834742 T7715 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2835300 T7715 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2835305 T7715 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2835307 T7715 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2835312 T7715 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2835332 T7715 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2835332 T7715 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1388944893121/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data/
[junit4] 2> 2835332 T7715 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c2bae18
[junit4] 2> 2835333 T7715 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data
[junit4] 2> 2835333 T7715 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data/index/
[junit4] 2> 2835334 T7715 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 2835334 T7715 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/control/data/index
[junit4] 2> 2835335 T7715 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=51.1748046875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=20.22639910826647, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.361328125, noCFSRatio=1.0
[junit4] 2> 2835336 T7715 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@4a1266b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5c49f02f; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
[junit4] 2> 2835336 T7715 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2835339 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2835339 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2835340 T7715 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2835340 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2835340 T7715 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2835340 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2835341 T7715 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2835341 T7715 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2835342 T7715 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2835342 T7715 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2835343 T7715 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2835343 T7715 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2835343 T7715 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2835344 T7715 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2835344 T7715 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2835345 T7715 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2835345 T7715 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2835361 T7715 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2835378 T7715 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2835378 T7715 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2835379 T7715 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5898393679458741]
[junit4] 2> 2835380 T7715 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@4a1266b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5c49f02f; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
[junit4] 2> 2835380 T7715 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2835380 T7715 oass.SolrIndexSearcher.<init> Opening Searcher@5278ff5e main
[junit4] 2> 2835383 T7716 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5278ff5e main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 2835385 T7715 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2835386 T7686 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
[junit4] 2> 2835386 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2835386 T7719 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44202 collection:control_collection shard:shard1
[junit4] 2> 2835388 T7719 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 2835388 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2835390 T7722 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11ce6ebf name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2835390 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2835392 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2835405 T7686 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2835406 T7719 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2835408 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2835409 T7719 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2835409 T7719 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2589 name=collection1 org.apache.solr.core.SolrCore@15dee390 url=http://127.0.0.1:44202/collection1 node=127.0.0.1:44202_ C2589_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:44202, core=collection1, node_name=127.0.0.1:44202_}
[junit4] 2> 2835409 T7719 C2589 P44202 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44202/collection1/
[junit4] 2> 2835410 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2835410 T7719 C2589 P44202 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2835410 T7719 C2589 P44202 oasc.SyncStrategy.syncToMe http://127.0.0.1:44202/collection1/ has no replicas
[junit4] 2> 2835410 T7719 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44202/collection1/ shard1
[junit4] 2> 2835411 T7719 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 2835413 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2835413 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2835413 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2835418 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2835421 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2835523 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2835523 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2835571 T7719 oasc.ZkController.register We are http://127.0.0.1:44202/collection1/ and leader is http://127.0.0.1:44202/collection1/
[junit4] 2> 2835571 T7719 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44202
[junit4] 2> 2835571 T7719 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2835572 T7719 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 2835572 T7719 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2835613 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2835613 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2835614 T7719 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2835614 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2835615 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2835615 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:44202",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:44202_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 2835682 T7686 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2835685 T7686 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56357
[junit4] 2> 2835686 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2835687 T7686 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2835687 T7686 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564
[junit4] 2> 2835687 T7686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/'
[junit4] 2> 2835704 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2835726 T7686 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/solr.xml
[junit4] 2> 2835807 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2835807 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2835818 T7686 oasc.CoreContainer.<init> New CoreContainer 1658576934
[junit4] 2> 2835819 T7686 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/]
[junit4] 2> 2835820 T7686 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2835821 T7686 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2835821 T7686 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2835822 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2835822 T7686 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2835822 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2835823 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2835823 T7686 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2835824 T7686 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2835833 T7686 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2835833 T7686 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2835834 T7686 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2835834 T7686 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60363/solr
[junit4] 2> 2835835 T7686 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2835836 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2835875 T7734 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e8b90eb name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2835877 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2835900 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2835902 T7737 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ceeeab4 name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2835902 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2835948 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2836951 T7686 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56357_
[junit4] 2> 2836953 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56357_
[junit4] 2> 2836957 T7722 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2836957 T7737 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2836957 T7712 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2836979 T7738 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 2836979 T7738 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2836981 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2836981 T7738 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2836981 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2836981 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2836982 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2836983 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:56357",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:56357_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2836983 T7713 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 2836984 T7713 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2836987 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2837090 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2837090 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2837090 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2837982 T7738 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2837982 T7738 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1
[junit4] 2> 2837982 T7738 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2837983 T7738 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2837983 T7738 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2837984 T7738 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1/'
[junit4] 2> 2837985 T7738 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1/lib/README' to classloader
[junit4] 2> 2837986 T7738 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1/lib/classes/' to classloader
[junit4] 2> 2838048 T7738 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2838109 T7738 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2838210 T7738 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2838219 T7738 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2838764 T7738 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2838767 T7738 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2838769 T7738 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2838772 T7738 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2838790 T7738 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2838791 T7738 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ShardSplitTest-jetty1-1388944895564/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1/
[junit4] 2> 2838791 T7738 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c2bae18
[junit4] 2> 2838791 T7738 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1
[junit4] 2> 2838792 T7738 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1/index/
[junit4] 2> 2838792 T7738 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 2838793 T7738 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty1/index
[junit4] 2> 2838794 T7738 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=51.1748046875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=20.22639910826647, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.361328125, noCFSRatio=1.0
[junit4] 2> 2838794 T7738 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@381705c5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7d67c554; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
[junit4] 2> 2838795 T7738 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2838797 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2838797 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2838798 T7738 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2838798 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2838798 T7738 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2838798 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2838799 T7738 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2838799 T7738 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2838799 T7738 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2838800 T7738 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2838800 T7738 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2838800 T7738 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2838800 T7738 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2838801 T7738 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2838801 T7738 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2838801 T7738 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2838802 T7738 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2838812 T7738 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2838816 T7738 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2838816 T7738 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2838817 T7738 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5898393679458741]
[junit4] 2> 2838817 T7738 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@381705c5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7d67c554; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
[junit4] 2> 2838817 T7738 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2838818 T7738 oass.SolrIndexSearcher.<init> Opening Searcher@5a4b96db main
[junit4] 2> 2838820 T7739 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a4b96db main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 2838822 T7738 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2838823 T7686 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
[junit4] 2> 2838824 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2838823 T7742 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56357 collection:collection1 shard:shard1
[junit4] 2> 2838825 T7742 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2838844 T7742 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2838855 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2838855 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2838855 T7742 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2838855 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2838856 T7742 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2590 name=collection1 org.apache.solr.core.SolrCore@6f0b6c21 url=http://127.0.0.1:56357/collection1 node=127.0.0.1:56357_ C2590_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:56357, core=collection1, node_name=127.0.0.1:56357_}
[junit4] 2> 2838856 T7742 C2590 P56357 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56357/collection1/
[junit4] 2> 2838856 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2838856 T7742 C2590 P56357 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2838857 T7742 C2590 P56357 oasc.SyncStrategy.syncToMe http://127.0.0.1:56357/collection1/ has no replicas
[junit4] 2> 2838857 T7742 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56357/collection1/ shard1
[junit4] 2> 2838857 T7742 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2838859 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2838860 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2838860 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2838860 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2838863 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2838864 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2838866 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2838969 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2838969 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2838969 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2839016 T7742 oasc.ZkController.register We are http://127.0.0.1:56357/collection1/ and leader is http://127.0.0.1:56357/collection1/
[junit4] 2> 2839016 T7742 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56357
[junit4] 2> 2839016 T7742 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2839016 T7742 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 2839016 T7742 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2839017 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2839018 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2839018 T7742 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2839018 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2839019 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2839020 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:56357",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:56357_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 2839029 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2839096 T7686 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2839099 T7686 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38382
[junit4] 2> 2839100 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2839100 T7686 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2839101 T7686 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982
[junit4] 2> 2839101 T7686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/'
[junit4] 2> 2839131 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2839131 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2839131 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2839141 T7686 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/solr.xml
[junit4] 2> 2839233 T7686 oasc.CoreContainer.<init> New CoreContainer 2064723613
[junit4] 2> 2839234 T7686 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/]
[junit4] 2> 2839236 T7686 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2839236 T7686 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2839237 T7686 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2839237 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2839238 T7686 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2839238 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2839238 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2839239 T7686 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2839239 T7686 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2839251 T7686 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2839251 T7686 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2839252 T7686 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2839252 T7686 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60363/solr
[junit4] 2> 2839253 T7686 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2839254 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2839256 T7754 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@172a9306 name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2839257 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2839261 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2839267 T7757 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a88a8f6 name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2839268 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2839272 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2840275 T7686 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38382_
[junit4] 2> 2840277 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38382_
[junit4] 2> 2840281 T7722 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2840282 T7757 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2840281 T7737 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2840281 T7712 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2840298 T7758 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 2840298 T7758 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2840299 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2840299 T7758 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2840300 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2840300 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2840301 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2840301 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38382",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38382_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2840302 T7713 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 2840302 T7713 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 2840305 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2840408 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2840408 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2840408 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2840408 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2841301 T7758 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2841301 T7758 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1
[junit4] 2> 2841301 T7758 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2841302 T7758 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2841303 T7758 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2841303 T7758 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1/'
[junit4] 2> 2841305 T7758 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1/lib/README' to classloader
[junit4] 2> 2841306 T7758 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1/lib/classes/' to classloader
[junit4] 2> 2841370 T7758 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2841442 T7758 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2841543 T7758 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2841553 T7758 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2841996 T7758 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2841999 T7758 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2842000 T7758 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2842004 T7758 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2842018 T7758 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2842018 T7758 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1388944898982/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2/
[junit4] 2> 2842018 T7758 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c2bae18
[junit4] 2> 2842019 T7758 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2
[junit4] 2> 2842019 T7758 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2/index/
[junit4] 2> 2842019 T7758 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 2842020 T7758 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty2/index
[junit4] 2> 2842021 T7758 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=51.1748046875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=20.22639910826647, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.361328125, noCFSRatio=1.0
[junit4] 2> 2842021 T7758 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@4fdd011 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5289967c; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
[junit4] 2> 2842022 T7758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2842024 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2842025 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2842026 T7758 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2842026 T7758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2842026 T7758 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2842027 T7758 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2842027 T7758 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2842027 T7758 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2842027 T7758 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2842028 T7758 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2842028 T7758 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2842029 T7758 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2842029 T7758 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2842040 T7758 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2842043 T7758 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2842044 T7758 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2842044 T7758 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5898393679458741]
[junit4] 2> 2842044 T7758 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@4fdd011 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5289967c; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
[junit4] 2> 2842045 T7758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2842045 T7758 oass.SolrIndexSearcher.<init> Opening Searcher@2f5b1603 main
[junit4] 2> 2842047 T7759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f5b1603 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 2842049 T7758 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2842049 T7686 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
[junit4] 2> 2842050 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2842049 T7762 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38382 collection:collection1 shard:shard2
[junit4] 2> 2842051 T7762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 2842067 T7762 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 2842069 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2842069 T7762 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2842069 T7762 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 2842069 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C2591 name=collection1 org.apache.solr.core.SolrCore@3327f008 url=http://127.0.0.1:38382/collection1 node=127.0.0.1:38382_ C2591_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:38382, core=collection1, node_name=127.0.0.1:38382_}
[junit4] 2> 2842069 T7762 C2591 P38382 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38382/collection1/
[junit4] 2> 2842070 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2842070 T7762 C2591 P38382 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2842070 T7762 C2591 P38382 oasc.SyncStrategy.syncToMe http://127.0.0.1:38382/collection1/ has no replicas
[junit4] 2> 2842070 T7762 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38382/collection1/ shard2
[junit4] 2> 2842070 T7762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 2842071 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2842074 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2842075 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842075 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842075 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842075 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842077 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2842080 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2842183 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842183 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842183 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842183 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842228 T7762 oasc.ZkController.register We are http://127.0.0.1:38382/collection1/ and leader is http://127.0.0.1:38382/collection1/
[junit4] 2> 2842228 T7762 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38382
[junit4] 2> 2842228 T7762 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2842228 T7762 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 2842228 T7762 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2842230 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2842230 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2842230 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2842230 T7762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2842231 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2842231 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:38382",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38382_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 2842232 T7686 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2842234 T7686 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45611
[junit4] 2> 2842234 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2842235 T7686 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2842235 T7686 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2842235 T7686 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208
[junit4] 2> 2842236 T7686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/'
[junit4] 2> 2842262 T7686 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/solr.xml
[junit4] 2> 2842323 T7686 oasc.CoreContainer.<init> New CoreContainer 1245378889
[junit4] 2> 2842323 T7686 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/]
[junit4] 2> 2842324 T7686 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2842325 T7686 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2842325 T7686 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2842325 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2842326 T7686 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2842326 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2842326 T7686 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2842326 T7686 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2842327 T7686 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2842334 T7686 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2842335 T7686 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2842335 T7686 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2842335 T7686 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60363/solr
[junit4] 2> 2842336 T7686 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2842336 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2842338 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842338 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842338 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842338 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2842340 T7774 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56c01a65 name:ZooKeeperConnection Watcher:127.0.0.1:60363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2842340 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2842351 T7686 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2842353 T7777 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23b0b33d name:ZooKeeperConnection Watcher:127.0.0.1:60363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2842354 T7686 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2842363 T7686 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2843366 T7686 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45611_
[junit4] 2> 2843394 T7686 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45611_
[junit4] 2> 2843397 T7777 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2843398 T7737 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2843398 T7712 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2843398 T7757 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2843397 T7722 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2843415 T7778 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 2843415 T7778 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2843416 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2843416 T7778 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2843417 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2843417 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2843418 T7713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2843418 T7713 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45611",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:45611_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2843418 T7713 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 2843419 T7713 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2843422 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2843524 T7722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2843524 T7737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2843524 T7757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2843524 T7777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2843524 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2844418 T7778 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2844418 T7778 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1
[junit4] 2> 2844418 T7778 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2844419 T7778 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2844419 T7778 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2844420 T7778 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1/'
[junit4] 2> 2844421 T7778 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1/lib/classes/' to classloader
[junit4] 2> 2844422 T7778 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1/lib/README' to classloader
[junit4] 2> 2844466 T7778 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2844508 T7778 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2844610 T7778 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2844617 T7778 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2845020 T7778 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2845023 T7778 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2845025 T7778 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2845028 T7778 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2845043 T7778 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2845043 T7778 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1388944902208/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3/
[junit4] 2> 2845043 T7778 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c2bae18
[junit4] 2> 2845044 T7778 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3
[junit4] 2> 2845044 T7778 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3/index/
[junit4] 2> 2845044 T7778 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 2845046 T7778 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1388944892811/jetty3/index
[junit4] 2> 2845046 T7778 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=51.1748046875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=20.22639910826647, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.361328125, noCFSRatio=1.0
[junit4] 2> 2845047 T7778 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@69634371 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e64b14d; maxCacheMB=0.0107421875 maxMergeSizeMB=0.953125)),segFN=segments_1,generation=1}
[junit4] 2> 2845047 T7778 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2845050 T7778 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2845050 T7778 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2845050 T7778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2845051 T7778 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2845051 T7778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_s
[...truncated too long message...]
a:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 3169708 T7798 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 337466 T7685 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> 357525 T7685 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest:
[junit4] 2> 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:526)
[junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4] 2> 357526 T7685 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
[junit4] 2> 360533 T7685 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:526)
[junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=pl_PL, timezone=Africa/Sao_Tome
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=2,free=279178864,total=494141440
[junit4] 2> NOTE: All tests run in this JVM: [TestUniqueKeyFieldResource, RAMDirectoryFactoryTest, DistributedSuggestComponentTest, TestComponentsName, BasicZkTest, ShardRoutingCustomTest, CollectionsAPIDistributedZkTest, SpellingQueryConverterTest, DirectSolrConnectionTest, TestNumberUtils, SolrCmdDistributorTest, TestAnalyzedSuggestions, TestAddFieldRealTimeGet, TestSolrQueryParserDefaultOperatorResource, HighlighterTest, ChangedSchemaMergeTest, TestUpdate, RequiredFieldsTest, BlockDirectoryTest, SimplePostToolTest, TestRangeQuery, TestSolrDeletionPolicy2, AlternateDirectoryTest, TestIndexSearcher, IndexBasedSpellCheckerTest, AliasIntegrationTest, TestBinaryResponseWriter, AddBlockUpdateTest, TestClassNameShortening, TestFastLRUCache, DateFieldTest, TestStressReorder, OutputWriterTest, TestFoldingMultitermQuery, MinimalSchemaTest, TestCoreDiscovery, TestHashPartitioner, TestFieldTypeCollectionResource, TestOmitPositions, TestSolrQueryParserResource, ZkSolrClientTest, TestRTGBase, RequestHandlersTest, TestNRTOpen, StatsComponentTest, TimeZoneUtilsTest, BlockCacheTest, ConnectionManagerTest, TestAtomicUpdateErrorCases, TestSolrXmlPersistence, TestArbitraryIndexDir, StressHdfsTest, TestSweetSpotSimilarityFactory, TestBinaryField, TestStressRecovery, SolrPluginUtilsTest, TestZkChroot, SpellCheckCollatorTest, TestCSVLoader, DebugComponentTest, TestSolr4Spatial, TestDFRSimilarityFactory, XsltUpdateRequestHandlerTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestIBSimilarityFactory, OpenExchangeRatesOrgProviderTest, ChaosMonkeySafeLeaderTest, TestIndexingPerformance, TestCodecSupport, FieldAnalysisRequestHandlerTest, UniqFieldsUpdateProcessorFactoryTest, DocValuesMultiTest, TestSolrJ, TestQuerySenderListener, DistributedQueryComponentCustomSortTest, TestRandomMergePolicy, FieldFacetExtrasTest, DefaultValueUpdateProcessorTest, TestPhraseSuggestions, IndexSchemaRuntimeFieldTest, CachingDirectoryFactoryTest, TestDistribDocBasedVersion, StandardRequestHandlerTest, AbstractAnalyticsStatsTest, TestFastWriter, QueryFacetTest, TestFieldResource, BadIndexSchemaTest, BasicDistributedZk2Test, SyncSliceTest, LeaderElectionIntegrationTest, ClusterStateUpdateTest, TestRandomFaceting, TestDistributedGrouping, TestFaceting, TestRecovery, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestJoin, TestMultiCoreConfBootstrap, TestReload, TestCoreContainer, TestGroupingSearch, DirectUpdateHandlerTest, OverseerCollectionProcessorTest, CurrencyFieldOpenExchangeTest, SolrIndexSplitterTest, AnalysisAfterCoreReloadTest, SolrRequestParserTest, TestStressLucene, PolyFieldTest, SolrCoreCheckLockOnStartupTest, TestWordDelimiterFilterFactory, SortByFunctionTest, TestSolrDeletionPolicy1, LukeRequestHandlerTest, DirectSolrSpellCheckerTest, TestQueryTypes, TermsComponentTest, TestValueSourceCache, FastVectorHighlighterTest, TestJmxIntegration, TestCSVResponseWriter, JsonLoaderTest, TestPHPSerializedResponseWriter, SearchHandlerTest, HighlighterConfigTest, UpdateParamsTest, TestQuerySenderNoQuery, ResponseLogComponentTest, BadComponentTest, TestSearchPerf, TestFuzzyAnalyzedSuggestions, TestSolrCoreProperties, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPerFieldSimilarity, TestDefaultSimilarityFactory, TestFastOutputStream, URLClassifyProcessorTest, PreAnalyzedFieldTest, PrimUtilsTest, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SystemInfoHandlerTest, FileUtilsTest, SolrTestCaseJ4Test, TestDistributedMissingSort, ExpressionTest, AbstractAnalyticsFacetTest, FieldFacetTest, FunctionTest, AssignTest, DeleteReplicaTest, DeleteShardTest, MigrateRouteKeyTest, ShardSplitTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ShardSplitTest -Dtests.seed=2B6E400BDCDB9BD9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=pl_PL -Dtests.timezone=Africa/Sao_Tome -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | ShardSplitTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest:
[junit4] > 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at java.lang.Object.wait(Object.java:503)
[junit4] > at java.util.TimerThread.mainLoop(Timer.java:526)
[junit4] > at java.util.TimerThread.run(Timer.java:505)
[junit4] > at __randomizedtesting.SeedInfo.seed([2B6E400BDCDB9BD9]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=7986, name=Timer-351, state=WAITING, group=TGRP-ShardSplitTest]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at java.lang.Object.wait(Object.java:503)
[junit4] > at java.util.TimerThread.mainLoop(Timer.java:526)
[junit4] > at java.util.TimerThread.run(Timer.java:505)
[junit4] > at __randomizedtesting.SeedInfo.seed([2B6E400BDCDB9BD9]:0)
[junit4] Completed on J1 in 360.59s, 1 test, 2 errors <<< FAILURES!
[...truncated 13 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:460: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:433: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:491: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1304: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:937: There were test failures: 358 suites, 1577 tests, 2 suite-level errors, 33 ignored (4 assumptions)
Total time: 116 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 485 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/485/
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.testDistribSearch
Error Message:
There were expected update fails expected:<0> but was:<1>
Stack Trace:
java.lang.AssertionError: There were expected update fails expected:<0> but was:<1>
at __randomizedtesting.SeedInfo.seed([7D9BA0CB76849935:FC7D2ED301DBF909]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:202)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:849)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 10879 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
[junit4] 2> 2861405 T10314 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 2861412 T10314 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyNothingIsSafeTest-1388838844789
[junit4] 2> 2861414 T10314 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 2861415 T10315 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 2861516 T10314 oasc.ZkTestServer.run start zk server on port:47879
[junit4] 2> 2861517 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2861526 T10322 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b38abf4 name:ZooKeeperConnection Watcher:127.0.0.1:47879 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2861527 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2861527 T10314 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2861542 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2861544 T10325 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c67ec9 name:ZooKeeperConnection Watcher:127.0.0.1:47879/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2861544 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2861545 T10314 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 2861555 T10314 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 2861558 T10314 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 2861560 T10314 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 2861568 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2861569 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 2861583 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 2861584 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 2861687 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2861688 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2861703 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2861704 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 2861707 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2861708 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 2861711 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2861712 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 2861715 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2861716 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 2861733 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2861734 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 2861737 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2861738 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2861741 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2861742 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 2861744 T10314 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2861745 T10314 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 2862067 T10314 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2862073 T10314 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:17199
[junit4] 2> 2862073 T10314 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2862074 T10314 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2862074 T10314 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1388838845128
[junit4] 2> 2862075 T10314 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1388838845128/'
[junit4] 2> 2862122 T10314 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1388838845128/solr.xml
[junit4] 2> 2862237 T10314 oasc.CoreContainer.<init> New CoreContainer 781904994
[junit4] 2> 2862238 T10314 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1388838845128/]
[junit4] 2> 2862241 T10314 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2862241 T10314 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2862242 T10314 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2862243 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2862243 T10314 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2862244 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2862244 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2862244 T10314 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2862245 T10314 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2862257 T10314 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2862258 T10314 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2862258 T10314 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2862259 T10314 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47879/solr
[junit4] 2> 2862259 T10314 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2862261 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2862300 T10337 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39741a47 name:ZooKeeperConnection Watcher:127.0.0.1:47879 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2862301 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2862304 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2862312 T10340 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24849279 name:ZooKeeperConnection Watcher:127.0.0.1:47879/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2862312 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2862314 T10314 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2862325 T10314 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 2862328 T10314 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 2862330 T10314 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:17199_
[junit4] 2> 2862338 T10314 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:17199_
[junit4] 2> 2862341 T10314 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 2862343 T10314 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 2862353 T10314 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2862362 T10314 oasc.Overseer.start Overseer (id=91018942532681731-127.0.0.1:17199_-n_0000000000) starting
[junit4] 2> 2862367 T10314 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 2862381 T10342 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2862382 T10314 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 2862384 T10314 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 2862386 T10314 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2862388 T10341 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2862395 T10343 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 2862395 T10343 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2862397 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2862397 T10343 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2862398 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2862398 T10341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:17199",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:17199_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2862399 T10341 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 2862399 T10341 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2862401 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2862409 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2863398 T10343 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2863398 T10343 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1388838845128/collection1
[junit4] 2> 2863398 T10343 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 2863399 T10343 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2863399 T10343 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 2863400 T10343 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1388838845128/collection1/'
[junit4] 2> 2863402 T10343 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1388838845128/collection1/lib/classes/' to classloader
[junit4] 2> 2863403 T10343 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1388838845128/collection1/lib/README' to classloader
[junit4] 2> 2863475 T10343 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2863546 T10343 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2863648 T10343 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2863677 T10343 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2864312 T10343 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2864317 T10343 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2864319 T10343 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2864325 T10343 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2864344 T10343 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2864345 T10343 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1388838845128/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/control/data/
[junit4] 2> 2864345 T10343 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@20449d82
[junit4] 2> 2864347 T10343 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/control/data
[junit4] 2> 2864348 T10343 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/control/data/index/
[junit4] 2> 2864349 T10343 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 2864350 T10343 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/control/data/index
[junit4] 2> 2864351 T10343 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=85.9794921875, floorSegmentMB=1.279296875, forceMergeDeletesPctAllowed=11.236400674380349, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2864368 T10343 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 2864368 T10343 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2864372 T10343 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2864372 T10343 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2864372 T10343 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2864373 T10343 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2864373 T10343 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2864373 T10343 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2864374 T10343 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2864374 T10343 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2864375 T10343 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2864375 T10343 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2864376 T10343 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2864376 T10343 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2864376 T10343 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2864377 T10343 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2864377 T10343 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2864378 T10343 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2864378 T10343 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2864401 T10343 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2864406 T10343 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2864406 T10343 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2864407 T10343 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=3, maxMergedSegmentMB=56.2314453125, floorSegmentMB=1.0439453125, forceMergeDeletesPctAllowed=2.973903554171339, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2864409 T10343 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 2864409 T10343 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2864410 T10343 oass.SolrIndexSearcher.<init> Opening Searcher@2c481afa main
[junit4] 2> 2864412 T10344 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c481afa main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 2864413 T10343 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2864414 T10314 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
[junit4] 2> 2864415 T10314 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2864414 T10347 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:17199 collection:control_collection shard:shard1
[junit4] 2> 2864417 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2864417 T10347 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 2864419 T10350 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ef23ddd name:ZooKeeperConnection Watcher:127.0.0.1:47879/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2864419 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2864421 T10314 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2864433 T10314 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2864440 T10347 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2864514 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2864515 T10347 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2864515 T10347 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3478 name=collection1 org.apache.solr.core.SolrCore@295fcbc9 url=http://127.0.0.1:17199/collection1 node=127.0.0.1:17199_ C3478_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:17199, core=collection1, node_name=127.0.0.1:17199_}
[junit4] 2> 2864515 T10347 C3478 P17199 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:17199/collection1/
[junit4] 2> 2864516 T10347 C3478 P17199 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2864516 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2864516 T10347 C3478 P17199 oasc.SyncStrategy.syncToMe http://127.0.0.1:17199/collection1/ has no replicas
[junit4] 2> 2864517 T10347 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:17199/collection1/ shard1
[junit4] 2> 2864517 T10347 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 2864524 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2864560 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2864560 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2864609 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2864618 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2864721 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2864721 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2864768 T10314 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2864773 T10314 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:15016
[junit4] 2> 2864774 T10314 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2864774 T10314 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2864775 T10314 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1388838847809
[junit4] 2> 2864775 T10314 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1388838847809/'
[junit4] 2> 2864812 T10347 oasc.ZkController.register We are http://127.0.0.1:17199/collection1/ and leader is http://127.0.0.1:17199/collection1/
[junit4] 2> 2864812 T10347 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:17199
[junit4] 2> 2864812 T10347 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2864813 T10347 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 2864813 T10347 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2864814 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2864815 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2864815 T10347 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2864815 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2864816 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2864817 T10341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:17199",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:17199_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 2864818 T10314 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1388838847809/solr.xml
[junit4] 2> 2864867 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2864918 T10314 oasc.CoreContainer.<init> New CoreContainer 1557972979
[junit4] 2> 2864919 T10314 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1388838847809/]
[junit4] 2> 2864921 T10314 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2864921 T10314 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2864922 T10314 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2864922 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2864923 T10314 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2864923 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2864923 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2864924 T10314 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2864924 T10314 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2864936 T10314 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2864937 T10314 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2864938 T10314 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2864938 T10314 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47879/solr
[junit4] 2> 2864939 T10314 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2864940 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2864947 T10362 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@527e4ea2 name:ZooKeeperConnection Watcher:127.0.0.1:47879 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2864949 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2864952 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2864954 T10365 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@441b506f name:ZooKeeperConnection Watcher:127.0.0.1:47879/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2864954 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2864975 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2864975 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2864976 T10314 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2865979 T10314 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:15016_
[junit4] 2> 2865996 T10314 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:15016_
[junit4] 2> 2865999 T10350 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2865999 T10365 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2865999 T10340 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2866017 T10366 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 2866017 T10366 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2866019 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2866019 T10366 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2866019 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2866020 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2866021 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2866021 T10341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:15016",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:15016_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2866022 T10341 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 2866022 T10341 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 2866024 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2866127 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2866127 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2866127 T10365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2867021 T10366 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2867021 T10366 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1388838847809/collection1
[junit4] 2> 2867021 T10366 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2867022 T10366 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2867022 T10366 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2867023 T10366 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1388838847809/collection1/'
[junit4] 2> 2867025 T10366 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1388838847809/collection1/lib/classes/' to classloader
[junit4] 2> 2867026 T10366 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1388838847809/collection1/lib/README' to classloader
[junit4] 2> 2867101 T10366 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2867186 T10366 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2867287 T10366 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2867315 T10366 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2867938 T10366 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2867942 T10366 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2867944 T10366 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2867950 T10366 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2867970 T10366 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2867970 T10366 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1388838847809/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty1/
[junit4] 2> 2867970 T10366 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@20449d82
[junit4] 2> 2867972 T10366 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty1
[junit4] 2> 2867973 T10366 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty1/index/
[junit4] 2> 2867974 T10366 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 2867976 T10366 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty1/index
[junit4] 2> 2867976 T10366 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=85.9794921875, floorSegmentMB=1.279296875, forceMergeDeletesPctAllowed=11.236400674380349, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2867992 T10366 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 2867992 T10366 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2867996 T10366 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2867997 T10366 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2867997 T10366 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2867997 T10366 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2867998 T10366 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2867998 T10366 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2867999 T10366 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2867999 T10366 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2867999 T10366 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2868000 T10366 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2868001 T10366 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2868001 T10366 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2868001 T10366 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2868002 T10366 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2868002 T10366 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2868003 T10366 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2868003 T10366 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2868022 T10366 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2868027 T10366 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2868027 T10366 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2868028 T10366 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=3, maxMergedSegmentMB=56.2314453125, floorSegmentMB=1.0439453125, forceMergeDeletesPctAllowed=2.973903554171339, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2868030 T10366 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 2868031 T10366 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2868031 T10366 oass.SolrIndexSearcher.<init> Opening Searcher@2e32e47 main
[junit4] 2> 2868035 T10367 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2e32e47 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 2868037 T10366 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2868038 T10314 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
[junit4] 2> 2868039 T10314 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2868038 T10370 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:15016 collection:collection1 shard:shard2
[junit4] 2> 2868040 T10370 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 2868061 T10370 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 2868070 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868070 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868070 T10370 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2868071 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868071 T10370 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3479 name=collection1 org.apache.solr.core.SolrCore@40cc75e3 url=http://127.0.0.1:15016/collection1 node=127.0.0.1:15016_ C3479_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:15016, core=collection1, node_name=127.0.0.1:15016_}
[junit4] 2> 2868071 T10370 C3479 P15016 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:15016/collection1/
[junit4] 2> 2868071 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2868072 T10370 C3479 P15016 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2868072 T10370 C3479 P15016 oasc.SyncStrategy.syncToMe http://127.0.0.1:15016/collection1/ has no replicas
[junit4] 2> 2868072 T10370 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:15016/collection1/ shard2
[junit4] 2> 2868073 T10370 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 2868075 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868086 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2868086 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2868086 T10365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2868097 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868098 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2868110 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868213 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2868213 T10365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2868213 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2868250 T10370 oasc.ZkController.register We are http://127.0.0.1:15016/collection1/ and leader is http://127.0.0.1:15016/collection1/
[junit4] 2> 2868250 T10370 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:15016
[junit4] 2> 2868250 T10370 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2868251 T10370 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 2868251 T10370 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2868253 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868253 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868253 T10370 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2868253 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868255 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2868256 T10341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:15016",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:15016_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 2868264 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2868345 T10314 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2868350 T10314 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:61310
[junit4] 2> 2868351 T10314 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2868352 T10314 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2868352 T10314 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1388838851415
[junit4] 2> 2868353 T10314 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1388838851415/'
[junit4] 2> 2868376 T10365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2868376 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2868376 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2868397 T10314 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1388838851415/solr.xml
[junit4] 2> 2868501 T10314 oasc.CoreContainer.<init> New CoreContainer 1601828885
[junit4] 2> 2868502 T10314 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1388838851415/]
[junit4] 2> 2868504 T10314 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2868504 T10314 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2868504 T10314 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2868505 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2868505 T10314 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2868506 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2868506 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2868507 T10314 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2868507 T10314 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2868519 T10314 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2868520 T10314 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2868520 T10314 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2868521 T10314 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47879/solr
[junit4] 2> 2868521 T10314 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2868523 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2868525 T10382 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@422458a9 name:ZooKeeperConnection Watcher:127.0.0.1:47879 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2868527 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2868530 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2868539 T10385 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f67da0d name:ZooKeeperConnection Watcher:127.0.0.1:47879/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2868539 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2868549 T10314 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2869552 T10314 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61310_
[junit4] 2> 2869555 T10314 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:61310_
[junit4] 2> 2869558 T10350 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2869558 T10340 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2869558 T10385 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2869558 T10365 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2869595 T10386 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 2869596 T10386 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2869598 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2869598 T10386 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2869598 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2869599 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2869600 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2869601 T10341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:61310",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:61310_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2869601 T10341 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 2869601 T10341 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2869604 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2869707 T10365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2869707 T10385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2869707 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2869707 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2870599 T10386 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2870599 T10386 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1388838851415/collection1
[junit4] 2> 2870599 T10386 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2870600 T10386 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2870600 T10386 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2870601 T10386 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1388838851415/collection1/'
[junit4] 2> 2870603 T10386 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1388838851415/collection1/lib/README' to classloader
[junit4] 2> 2870604 T10386 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1388838851415/collection1/lib/classes/' to classloader
[junit4] 2> 2870677 T10386 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2870745 T10386 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2870846 T10386 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2870874 T10386 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2871499 T10386 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2871504 T10386 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2871506 T10386 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2871511 T10386 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2871531 T10386 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2871531 T10386 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1388838851415/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty2/
[junit4] 2> 2871532 T10386 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@20449d82
[junit4] 2> 2871533 T10386 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty2
[junit4] 2> 2871534 T10386 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty2/index/
[junit4] 2> 2871535 T10386 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 2871538 T10386 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty2/index
[junit4] 2> 2871540 T10386 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=85.9794921875, floorSegmentMB=1.279296875, forceMergeDeletesPctAllowed=11.236400674380349, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2871545 T10386 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 2871545 T10386 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2871549 T10386 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2871549 T10386 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2871550 T10386 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2871550 T10386 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2871550 T10386 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2871551 T10386 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2871551 T10386 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2871552 T10386 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2871552 T10386 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2871553 T10386 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2871553 T10386 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2871553 T10386 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2871554 T10386 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2871554 T10386 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2871555 T10386 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2871555 T10386 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2871556 T10386 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2871576 T10386 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2871581 T10386 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2871582 T10386 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2871583 T10386 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=3, maxMergedSegmentMB=56.2314453125, floorSegmentMB=1.0439453125, forceMergeDeletesPctAllowed=2.973903554171339, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2871585 T10386 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 2871585 T10386 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2871585 T10386 oass.SolrIndexSearcher.<init> Opening Searcher@516bae18 main
[junit4] 2> 2871589 T10387 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@516bae18 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 2871605 T10386 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2871606 T10314 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
[junit4] 2> 2871606 T10314 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2871606 T10390 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:61310 collection:collection1 shard:shard1
[junit4] 2> 2871608 T10390 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2871622 T10390 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2871625 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2871626 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2871626 T10390 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2871626 T10390 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 2871626 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C3480 name=collection1 org.apache.solr.core.SolrCore@6b58093b url=http://127.0.0.1:61310/collection1 node=127.0.0.1:61310_ C3480_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:61310, core=collection1, node_name=127.0.0.1:61310_}
[junit4] 2> 2871626 T10390 C3480 P61310 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:61310/collection1/
[junit4] 2> 2871627 T10390 C3480 P61310 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2871627 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2871627 T10390 C3480 P61310 oasc.SyncStrategy.syncToMe http://127.0.0.1:61310/collection1/ has no replicas
[junit4] 2> 2871628 T10390 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:61310/collection1/ shard1
[junit4] 2> 2871628 T10390 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2871637 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2871649 T10385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871649 T10365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871649 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871649 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871663 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2871667 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2871772 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871772 T10365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871772 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871772 T10385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871815 T10390 oasc.ZkController.register We are http://127.0.0.1:61310/collection1/ and leader is http://127.0.0.1:61310/collection1/
[junit4] 2> 2871815 T10390 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:61310
[junit4] 2> 2871815 T10390 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2871816 T10390 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 2871816 T10390 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2871817 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2871818 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2871818 T10390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2871818 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2871819 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2871820 T10341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:61310",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:61310_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 2871829 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2871918 T10314 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2871922 T10314 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13112
[junit4] 2> 2871923 T10314 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2871923 T10314 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2871924 T10314 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1388838854982
[junit4] 2> 2871924 T10314 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1388838854982/'
[junit4] 2> 2871932 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871932 T10385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871932 T10365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871932 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2871967 T10314 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1388838854982/solr.xml
[junit4] 2> 2872075 T10314 oasc.CoreContainer.<init> New CoreContainer 1416436662
[junit4] 2> 2872076 T10314 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1388838854982/]
[junit4] 2> 2872078 T10314 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2872078 T10314 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2872079 T10314 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2872079 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2872079 T10314 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2872080 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2872080 T10314 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2872081 T10314 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2872081 T10314 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2872093 T10314 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2872093 T10314 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2872094 T10314 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2872094 T10314 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47879/solr
[junit4] 2> 2872095 T10314 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2872096 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2872099 T10402 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@483d4317 name:ZooKeeperConnection Watcher:127.0.0.1:47879 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2872102 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2872104 T10314 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2872106 T10405 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ebb0f36 name:ZooKeeperConnection Watcher:127.0.0.1:47879/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2872107 T10314 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2872117 T10314 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2873120 T10314 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13112_
[junit4] 2> 2873122 T10314 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13112_
[junit4] 2> 2873125 T10350 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2873125 T10365 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2873125 T10405 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2873125 T10340 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2873125 T10385 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2873144 T10406 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 2873144 T10406 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2873146 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2873146 T10406 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2873146 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2873147 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2873148 T10341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2873148 T10341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:13112",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:13112_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2873148 T10341 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 2873149 T10341 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 2873152 T10340 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2873254 T10340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2873254 T10405 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2873254 T10365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2873254 T10350 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2873254 T10385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 2874147 T10406 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2874147 T10406 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1388838854982/collection1
[junit4] 2> 2874147 T10406 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2874148 T10406 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2874148 T10406 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2874149 T10406 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1388838854982/collection1/'
[junit4] 2> 2874151 T10406 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1388838854982/collection1/lib/classes/' to classloader
[junit4] 2> 2874152 T10406 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1388838854982/collection1/lib/README' to classloader
[junit4] 2> 2874232 T10406 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2874302 T10406 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2874404 T10406 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2874435 T10406 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2875076 T10406 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2875080 T10406 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2875083 T10406 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2875088 T10406 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2875108 T10406 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2875
[...truncated too long message...]
nc: core=collection1 url=http://127.0.0.1:27417 DONE. sync succeeded
[junit4] 2> 2966419 T10448 C3497 P27417 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
[junit4] 2> 2966420 T10448 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 2966420 T10448 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@2a4bd717
[junit4] 2> 2966422 T10448 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=13,cumulative_deletesById=5,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=1}
[junit4] 2> 2966423 T10448 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 2966423 T10448 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 2966423 T10448 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 2966426 T10448 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 2966426 T10448 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 2966426 T10448 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty5 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty5;done=false>>]
[junit4] 2> 2966427 T10448 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty5
[junit4] 2> 2966427 T10448 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty5/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty5/index;done=false>>]
[junit4] 2> 2966427 T10448 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1388838844788/jetty5/index
[junit4] 2> 2966427 T10448 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 2966428 T10448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2966428 T10448 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 2966428 T10448 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=hr, timezone=Europe/Istanbul
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=2,free=186692112,total=499646464
[junit4] 2> NOTE: All tests run in this JVM: [HardAutoCommitTest, RAMDirectoryFactoryTest, DirectSolrConnectionTest, TestRecovery, TestDefaultSearchFieldResource, TestSolrCoreProperties, MigrateRouteKeyTest, SolrIndexConfigTest, TestSolr4Spatial, TestSolrXml, TestQueryTypes, CurrencyFieldXmlFileTest, DefaultValueUpdateProcessorTest, TestIndexSearcher, FileUtilsTest, TestMultiCoreConfBootstrap, SimplePostToolTest, TestSchemaResource, DocumentAnalysisRequestHandlerTest, StatsComponentTest, BasicZkTest, TestLazyCores, TestDFRSimilarityFactory, TestFastLRUCache, TestSolrDeletionPolicy2, ZkNodePropsTest, TestCodecSupport, RequiredFieldsTest, TestComponentsName, TestDefaultSimilarityFactory, TestStressReorder, SignatureUpdateProcessorFactoryTest, DisMaxRequestHandlerTest, AddSchemaFieldsUpdateProcessorFactoryTest, IndexSchemaRuntimeFieldTest, TestFoldingMultitermQuery, TestSolrQueryParserDefaultOperatorResource, TestUtils, FieldAnalysisRequestHandlerTest, PathHierarchyTokenizerFactoryTest, LeaderElectionIntegrationTest, TestFuzzyAnalyzedSuggestions, XsltUpdateRequestHandlerTest, CurrencyFieldOpenExchangeTest, OutputWriterTest, BadCopyFieldTest, TimeZoneUtilsTest, FieldFacetExtrasTest, FieldFacetTest, TestRangeQuery, TermVectorComponentDistributedTest, DocValuesMultiTest, SolrTestCaseJ4Test, TestSchemaVersionResource, TestRandomFaceting, TestCSVResponseWriter, TestPartialUpdateDeduplication, TestOmitPositions, ConnectionManagerTest, TestSolrQueryParserResource, TestSimpleQParserPlugin, DistributedSuggestComponentTest, TestStressUserVersions, TestNRTOpen, TestPhraseSuggestions, ChangedSchemaMergeTest, TestClassNameShortening, TestFiltering, ChaosMonkeySafeLeaderTest, PrimUtilsTest, PreAnalyzedFieldTest, TestAddFieldRealTimeGet, TestDistribDocBasedVersion, TestWordDelimiterFilterFactory, AddBlockUpdateTest, SortByFunctionTest, TestStressLucene, TestFileDictionaryLookup, TestSerializedLuceneMatchVersion, TestManagedSchemaFieldResource, TestRTGBase, NoCacheHeaderTest, TestReload, XmlUpdateRequestHandlerTest, RequestHandlersTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OverseerTest, ShardRoutingTest, ClusterStateUpdateTest, ZkSolrClientTest, TestZkChroot, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, PolyFieldTest, SolrCoreCheckLockOnStartupTest, TestAtomicUpdateErrorCases, StatelessScriptUpdateProcessorFactoryTest, DistanceFunctionTest, DebugComponentTest, LukeRequestHandlerTest, TestReversedWildcardFilterFactory, DocumentBuilderTest, TermVectorComponentTest, MoreLikeThisHandlerTest, TestArbitraryIndexDir, SolrPluginUtilsTest, TestJmxIntegration, JSONWriterTest, UniqFieldsUpdateProcessorFactoryTest, CSVRequestHandlerTest, HighlighterConfigTest, TestQuerySenderListener, TestSolrIndexConfig, CopyFieldTest, BadComponentTest, TestMergePolicyConfig, MultiTermTest, TestBinaryField, NumericFieldsTest, MinimalSchemaTest, TestConfig, TestPostingsSolrHighlighter, NotRequiredUniqueKeyTest, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestNumberUtils, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TestBM25SimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, ResourceLoaderTest, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, PluginInfoTest, ChaosMonkeyNothingIsSafeTest]
[junit4] Completed on J1 in 105.71s, 1 test, 1 failure <<< FAILURES!
[...truncated 131 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:460: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:433: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:491: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1304: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:937: There were test failures: 358 suites, 1577 tests, 1 failure, 33 ignored (4 assumptions)
Total time: 115 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure