You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/02/17 12:07:23 UTC
[JENKINS] Lucene-Solr-4.x-Linux
(64bit/jrockit-jdk1.6.0_45-R28.2.7-4.1.0) - Build # 9397 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9397/
Java: 64bit/jrockit-jdk1.6.0_45-R28.2.7-4.1.0 -XnoOpt
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([487D3EBA84D183DE]:0)
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([487D3EBA84D183DE]:0)
Build Log:
[...truncated 10274 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> 206458 T553 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 206458 T553 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 206492 T553 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeySafeLeaderTest-1392627983956
[junit4] 2> 206492 T553 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 206493 T554 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 206593 T553 oasc.ZkTestServer.run start zk server on port:46335
[junit4] 2> 206594 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 206596 T560 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d320729 name:ZooKeeperConnection Watcher:127.0.0.1:46335 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 206596 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 206597 T553 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 206601 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 206601 T562 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d3243df name:ZooKeeperConnection Watcher:127.0.0.1:46335/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 206602 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 206602 T553 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 206606 T553 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 206610 T553 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 206612 T553 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 206616 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 206617 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 206623 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 206624 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 206629 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 206630 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 206633 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 206633 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 206637 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 206638 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 206641 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 206642 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 206645 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 206646 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 206691 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 206692 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 206696 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 206696 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 206700 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 206701 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 206705 T553 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 206706 T553 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 206712 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 206713 T564 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c24df05 name:ZooKeeperConnection Watcher:127.0.0.1:46335/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 206713 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 206794 T553 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 206796 T553 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
[junit4] 2> 206813 T553 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:35615
[junit4] 2> 206814 T553 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 206815 T553 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 206815 T553 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1392627984180
[junit4] 2> 206815 T553 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1392627984180/'
[junit4] 2> 206828 T553 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1392627984180/solr.xml
[junit4] 2> 206874 T553 oasc.CoreContainer.<init> New CoreContainer 477326165
[junit4] 2> 206874 T553 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1392627984180/]
[junit4] 2> 206875 T553 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 206876 T553 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 206876 T553 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 206876 T553 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 206877 T553 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 206877 T553 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 206877 T553 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 206877 T553 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 206878 T553 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 206881 T553 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 206881 T553 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 206882 T553 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 206882 T553 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46335/solr
[junit4] 2> 206882 T553 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 206883 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 206885 T575 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c77c0c2 name:ZooKeeperConnection Watcher:127.0.0.1:46335 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 206886 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 206890 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 206891 T577 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c77f398 name:ZooKeeperConnection Watcher:127.0.0.1:46335/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 206892 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 206895 T553 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 206902 T553 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 206911 T553 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 206916 T553 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35615_
[junit4] 2> 206918 T553 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35615_
[junit4] 2> 206926 T553 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 206931 T553 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 206941 T553 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:35615_
[junit4] 2> 206942 T553 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 206948 T553 oasc.Overseer.start Overseer (id=91267267556802564-127.0.0.1:35615_-n_0000000000) starting
[junit4] 2> 206955 T553 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 206963 T579 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 206966 T553 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 206966 T579 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 206974 T553 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 206979 T553 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 206990 T578 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 206994 T580 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 206994 T580 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 206999 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 207002 T580 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 207003 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 207005 T578 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:35615",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35615_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 207009 T578 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 207016 T578 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 207022 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 207024 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208002 T580 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 208003 T580 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1392627984180/collection1
[junit4] 2> 208003 T580 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 208004 T580 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 208004 T580 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 208006 T580 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 208006 T580 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1392627984180/collection1/'
[junit4] 2> 208006 T580 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1392627984180/collection1/lib/classes/' to classloader
[junit4] 2> 208007 T580 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1392627984180/collection1/lib/README' to classloader
[junit4] 2> 208052 T580 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 208108 T580 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 208110 T580 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 208124 T580 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 208577 T580 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 208580 T580 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 208582 T580 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 208586 T580 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 208624 T580 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 208624 T580 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1392627984180/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/control/data/
[junit4] 2> 208625 T580 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ca148ca
[junit4] 2> 208625 T580 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/control/data
[junit4] 2> 208626 T580 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/control/data/index/
[junit4] 2> 208626 T580 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 208627 T580 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/control/data/index
[junit4] 2> 208627 T580 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 208630 T580 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 208630 T580 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 208632 T580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 208641 T580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 208642 T580 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 208642 T580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 208642 T580 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 208642 T580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 208643 T580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 208643 T580 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 208644 T580 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 208644 T580 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 208645 T580 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 208645 T580 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 208645 T580 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 208646 T580 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 208646 T580 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 208647 T580 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 208649 T580 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 208651 T580 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 208652 T580 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 208652 T580 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 208653 T580 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 208653 T580 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 208654 T580 oass.SolrIndexSearcher.<init> Opening Searcher@1c2af238[collection1] main
[junit4] 2> 208657 T581 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c2af238[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 208658 T580 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 208658 T584 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:35615 collection:control_collection shard:shard1
[junit4] 2> 208658 T553 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 208659 T553 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 208660 T584 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 208667 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 208668 T586 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c2f5445 name:ZooKeeperConnection Watcher:127.0.0.1:46335/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 208668 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 208668 T584 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 208670 T553 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 208671 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208673 T584 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 208673 T584 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C53 name=collection1 org.apache.solr.core.SolrCore@1c2634dd url=https://127.0.0.1:35615/collection1 node=127.0.0.1:35615_ C53_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:35615, core=collection1, node_name=127.0.0.1:35615_}
[junit4] 2> 208674 T584 C53 P35615 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:35615/collection1/
[junit4] 2> 208675 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 208676 T584 C53 P35615 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 208677 T553 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 208678 T584 C53 P35615 oasc.SyncStrategy.syncToMe https://127.0.0.1:35615/collection1/ has no replicas
[junit4] 2> 208678 T584 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:35615/collection1/ shard1
[junit4] 2> 208678 T584 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 208679 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208680 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208680 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208690 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208693 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 208697 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208761 T553 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 208763 T553 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 208766 T553 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
[junit4] 2> 208767 T553 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:32932
[junit4] 2> 208768 T553 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 208768 T553 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 208769 T553 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1392627986143
[junit4] 2> 208769 T553 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1392627986143/'
[junit4] 2> 208782 T553 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1392627986143/solr.xml
[junit4] 2> 208801 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208801 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208830 T553 oasc.CoreContainer.<init> New CoreContainer 472108688
[junit4] 2> 208831 T553 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1392627986143/]
[junit4] 2> 208832 T553 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 208832 T553 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 208833 T553 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 208833 T553 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 208833 T553 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 208834 T553 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 208834 T553 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 208834 T553 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 208834 T553 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 208838 T553 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 208838 T553 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 208839 T553 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 208839 T553 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46335/solr
[junit4] 2> 208839 T553 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 208840 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 208841 T597 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c8d1d29 name:ZooKeeperConnection Watcher:127.0.0.1:46335 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 208842 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 208842 T584 oasc.ZkController.register We are https://127.0.0.1:35615/collection1/ and leader is https://127.0.0.1:35615/collection1/
[junit4] 2> 208842 T584 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:35615
[junit4] 2> 208843 T584 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 208843 T584 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 208843 T584 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 208845 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208845 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 208845 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208846 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208846 T599 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c8d503d name:ZooKeeperConnection Watcher:127.0.0.1:46335/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 208846 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 208847 T584 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 208849 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 208850 T578 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:35615",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35615_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 208855 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 208855 T553 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 208960 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208960 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 208961 T599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 209863 T553 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32932_
[junit4] 2> 209866 T553 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32932_
[junit4] 2> 209873 T586 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 209874 T577 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 209874 T599 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 209890 T600 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 209891 T600 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 209893 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 209893 T600 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 209894 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 209894 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 209899 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 209900 T578 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:32932",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:32932_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 209901 T578 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 209902 T578 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 210078 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 210181 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 210181 T599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 210181 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 210894 T600 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 210894 T600 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1392627986143/collection1
[junit4] 2> 210895 T600 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 210896 T600 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 210896 T600 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 210897 T600 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 210898 T600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1392627986143/collection1/'
[junit4] 2> 210898 T600 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1392627986143/collection1/lib/classes/' to classloader
[junit4] 2> 210899 T600 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1392627986143/collection1/lib/README' to classloader
[junit4] 2> 210946 T600 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 211004 T600 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 211006 T600 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 211014 T600 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 211458 T600 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 211461 T600 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 211463 T600 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 211467 T600 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 211496 T600 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 211497 T600 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1392627986143/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty1/
[junit4] 2> 211497 T600 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ca148ca
[junit4] 2> 211498 T600 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty1
[junit4] 2> 211498 T600 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty1/index/
[junit4] 2> 211498 T600 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 211498 T600 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty1/index
[junit4] 2> 211499 T600 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 211509 T600 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 211510 T600 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 211513 T600 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 211513 T600 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 211513 T600 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 211513 T600 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 211514 T600 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 211514 T600 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 211514 T600 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 211514 T600 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 211514 T600 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 211515 T600 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 211516 T600 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 211516 T600 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 211516 T600 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 211517 T600 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 211517 T600 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 211518 T600 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 211520 T600 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 211522 T600 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 211523 T600 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 211524 T600 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 211525 T600 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 211525 T600 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 211525 T600 oass.SolrIndexSearcher.<init> Opening Searcher@1c3c9a67[collection1] main
[junit4] 2> 211531 T601 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c3c9a67[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 211533 T600 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 211534 T604 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:32932 collection:collection1 shard:shard1
[junit4] 2> 211534 T553 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 211534 T553 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 211535 T604 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 211544 T604 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 211547 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211547 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211547 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211548 T604 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 211548 T604 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C54 name=collection1 org.apache.solr.core.SolrCore@1c33b14a url=https://127.0.0.1:32932/collection1 node=127.0.0.1:32932_ C54_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:32932, core=collection1, node_name=127.0.0.1:32932_}
[junit4] 2> 211548 T604 C54 P32932 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:32932/collection1/
[junit4] 2> 211548 T604 C54 P32932 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 211549 T604 C54 P32932 oasc.SyncStrategy.syncToMe https://127.0.0.1:32932/collection1/ has no replicas
[junit4] 2> 211549 T604 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:32932/collection1/ shard1
[junit4] 2> 211549 T604 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 211549 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 211555 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211559 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211563 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211615 T553 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 211615 T553 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 211618 T553 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
[junit4] 2> 211619 T553 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55659
[junit4] 2> 211620 T553 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 211620 T553 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 211620 T553 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1392627989000
[junit4] 2> 211621 T553 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1392627989000/'
[junit4] 2> 211634 T553 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1392627989000/solr.xml
[junit4] 2> 211667 T599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 211667 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 211667 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 211682 T553 oasc.CoreContainer.<init> New CoreContainer 480093852
[junit4] 2> 211682 T553 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1392627989000/]
[junit4] 2> 211684 T553 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 211684 T553 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 211684 T553 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 211684 T553 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 211685 T553 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 211685 T553 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 211685 T553 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 211686 T553 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 211686 T553 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 211689 T553 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 211690 T553 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 211690 T553 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 211691 T553 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46335/solr
[junit4] 2> 211691 T553 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 211692 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211694 T615 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c8d4d98 name:ZooKeeperConnection Watcher:127.0.0.1:46335 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211694 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211697 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 211698 T617 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c8e5c43 name:ZooKeeperConnection Watcher:127.0.0.1:46335/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 211698 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 211704 T553 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 211710 T604 oasc.ZkController.register We are https://127.0.0.1:32932/collection1/ and leader is https://127.0.0.1:32932/collection1/
[junit4] 2> 211710 T604 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:32932
[junit4] 2> 211710 T604 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 211711 T604 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 211711 T604 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 211712 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211713 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211713 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211713 T604 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 211715 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 211716 T578 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:32932",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:32932_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 211719 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 211822 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 211822 T599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 211822 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 211822 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 212709 T553 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55659_
[junit4] 2> 212710 T553 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55659_
[junit4] 2> 212714 T617 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 212714 T599 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 212715 T586 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 212716 T577 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 212723 T618 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 212723 T618 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 212725 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 212725 T618 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 212725 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 212725 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 212727 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 212728 T578 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:55659",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:55659_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 212728 T578 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 212728 T578 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 212731 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 212836 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 212836 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 212836 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 212836 T599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 213725 T618 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 213726 T618 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1392627989000/collection1
[junit4] 2> 213726 T618 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 213727 T618 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 213727 T618 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 213729 T618 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 213729 T618 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1392627989000/collection1/'
[junit4] 2> 213729 T618 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1392627989000/collection1/lib/classes/' to classloader
[junit4] 2> 213730 T618 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1392627989000/collection1/lib/README' to classloader
[junit4] 2> 213778 T618 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 213830 T618 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 213832 T618 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 213841 T618 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 214448 T618 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 214453 T618 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 214456 T618 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 214461 T618 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 214496 T618 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 214497 T618 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1392627989000/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty2/
[junit4] 2> 214497 T618 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ca148ca
[junit4] 2> 214498 T618 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty2
[junit4] 2> 214498 T618 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty2/index/
[junit4] 2> 214498 T618 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 214498 T618 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty2/index
[junit4] 2> 214499 T618 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 214558 T618 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 214558 T618 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 214562 T618 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 214562 T618 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 214562 T618 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 214562 T618 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 214563 T618 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 214563 T618 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 214563 T618 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 214563 T618 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 214564 T618 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 214564 T618 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 214565 T618 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 214565 T618 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 214566 T618 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 214566 T618 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 214567 T618 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 214567 T618 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 214569 T618 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 214572 T618 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 214572 T618 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 214573 T618 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 214574 T618 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 214575 T618 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 214575 T618 oass.SolrIndexSearcher.<init> Opening Searcher@1c3bd948[collection1] main
[junit4] 2> 214580 T619 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c3bd948[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 214583 T618 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 214584 T553 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 214584 T553 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 214588 T622 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:55659 collection:collection1 shard:shard2
[junit4] 2> 214591 T622 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 214601 T622 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 214604 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 214604 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 214604 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 214605 T622 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 214605 T622 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C55 name=collection1 org.apache.solr.core.SolrCore@1c273065 url=https://127.0.0.1:55659/collection1 node=127.0.0.1:55659_ C55_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:55659, core=collection1, node_name=127.0.0.1:55659_}
[junit4] 2> 214605 T622 C55 P55659 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:55659/collection1/
[junit4] 2> 214606 T622 C55 P55659 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 214606 T622 C55 P55659 oasc.SyncStrategy.syncToMe https://127.0.0.1:55659/collection1/ has no replicas
[junit4] 2> 214607 T622 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:55659/collection1/ shard2
[junit4] 2> 214607 T622 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 214607 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 214614 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 214620 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214620 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214620 T599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214621 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214631 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 214636 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 214683 T553 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 214684 T553 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 214686 T553 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
[junit4] 2> 214687 T553 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:34773
[junit4] 2> 214689 T553 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 214689 T553 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 214690 T553 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1392627992050
[junit4] 2> 214690 T553 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1392627992050/'
[junit4] 2> 214704 T553 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1392627992050/solr.xml
[junit4] 2> 214744 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214744 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214744 T599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214745 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214775 T553 oasc.CoreContainer.<init> New CoreContainer 474735974
[junit4] 2> 214776 T553 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1392627992050/]
[junit4] 2> 214777 T553 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 214777 T553 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 214777 T622 oasc.ZkController.register We are https://127.0.0.1:55659/collection1/ and leader is https://127.0.0.1:55659/collection1/
[junit4] 2> 214778 T553 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 214778 T622 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55659
[junit4] 2> 214778 T553 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 214778 T622 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 214779 T553 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 214779 T622 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 214779 T553 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 214779 T622 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 214780 T553 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 214780 T553 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 214781 T553 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 214781 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 214782 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 214782 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 214783 T622 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 214784 T553 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 214785 T553 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 214785 T553 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 214785 T553 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46335/solr
[junit4] 2> 214786 T553 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 214786 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 214786 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 214787 T578 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:55659",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:55659_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 214788 T633 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c52b4ec name:ZooKeeperConnection Watcher:127.0.0.1:46335 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 214789 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 214792 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 214796 T553 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 214797 T635 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c52e7c8 name:ZooKeeperConnection Watcher:127.0.0.1:46335/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 214797 T553 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 214803 T553 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 214895 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214895 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214896 T635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214896 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 214895 T599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 215808 T553 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34773_
[junit4] 2> 215810 T553 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34773_
[junit4] 2> 215814 T617 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 215815 T577 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 215815 T586 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 215816 T635 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 215819 T599 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 215827 T636 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 215827 T636 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 215829 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 215829 T636 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 215829 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 215830 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 215832 T578 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 215833 T578 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34773",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34773_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 215834 T578 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 215834 T578 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 215838 T577 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 215941 T617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 215941 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 215941 T635 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 215941 T599 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 215942 T577 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 216830 T636 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 216830 T636 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1392627992050/collection1
[junit4] 2> 216830 T636 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 216831 T636 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 216831 T636 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 216833 T636 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 216833 T636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1392627992050/collection1/'
[junit4] 2> 216833 T636 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1392627992050/collection1/lib/classes/' to classloader
[junit4] 2> 216834 T636 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1392627992050/collection1/lib/README' to classloader
[junit4] 2> 216880 T636 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 216931 T636 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 216934 T636 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 216943 T636 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 217383 T636 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 217386 T636 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 217388 T636 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 217392 T636 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 217423 T636 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 217423 T636 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1392627992050/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty3/
[junit4] 2> 217424 T636 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ca148ca
[junit4] 2> 217424 T636 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty3
[junit4] 2> 217425 T636 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty3/index/
[junit4] 2> 217425 T636 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 217425 T636 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty3/index
[junit4] 2> 217426 T636 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 217428 T636 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392627983956/jetty3/index,segFN=segments_1,generation=1}
[junit4] 2> 217428 T636 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 217431 T636 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 217431 T636 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 217432 T636 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 217432 T636 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 217432 T636 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 217432 T636 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 217433 T636 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 217433 T636 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 217433 T636 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 217434 T636 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 217435 T636 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 217435 T636 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 217435 T636 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 217436 T636 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 217436 T636 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 217437 T636 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 217439 T636 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 217441 T636 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 217442 T636 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 217443 T636 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 217443 T636 oasc.SolrDeletionPolicy.onInit
[...truncated too long message...]
[junit4] 2> at org.eclipse.jetty.server.Server.doStop(Server.java:338)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:455)
[junit4] 2> at org.apache.solr.cloud.ChaosMonkey.stopJettySolrRunner(ChaosMonkey.java:227)
[junit4] 2> at org.apache.solr.cloud.ChaosMonkey.stop(ChaosMonkey.java:563)
[junit4] 2> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.destroyServers(AbstractFullDistribZkTestBase.java:1586)
[junit4] 2> at org.apache.solr.BaseDistributedSearchTestCase.tearDown(BaseDistributedSearchTestCase.java:288)
[junit4] 2> at org.apache.solr.cloud.AbstractDistribZkTestBase.tearDown(AbstractDistribZkTestBase.java:222)
[junit4] 2> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.tearDown(AbstractFullDistribZkTestBase.java:1567)
[junit4] 2> at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.tearDown(ChaosMonkeySafeLeaderTest.java:84)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:597)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:795)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2> 60) Thread[id=1145, name=qtp475081948-1145 Acceptor0 SslSelectChannelConnector@127.0.0.1:47713, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:102)
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2> 7431940 T731 oaz.ClientCnxn$SendThread.run WARN Session 0x1443f18a6580015 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=vi_VN, timezone=SystemV/MST7MDT
[junit4] 2> NOTE: Linux 3.8.0-35-generic amd64/Oracle Corporation 1.6.0_45 (64-bit)/cpus=8,threads=63,free=137171096,total=346308608
[junit4] 2> NOTE: All tests run in this JVM: [ChangedSchemaMergeTest, SignatureUpdateProcessorFactoryTest, TestFileDictionaryLookup, TestFastWriter, AssignTest, PreAnalyzedUpdateProcessorTest, TestIndexingPerformance, BlockDirectoryTest, TestFastOutputStream, TestCSVLoader, TestHashPartitioner, ZkSolrClientTest, HardAutoCommitTest, DistributedSuggestComponentTest, SuggesterWFSTTest, TestNumberUtils, TimeZoneUtilsTest, TestDistributedMissingSort, ConnectionManagerTest, RAMDirectoryFactoryTest, TestNonDefinedSimilarityFactory, TestInfoStreamLogging, TestSolrXmlPersistence, TestQuerySenderListener, BasicZkTest, MinimalSchemaTest, ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=487D3EBA84D183DE -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=vi_VN -Dtests.timezone=SystemV/MST7MDT -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([487D3EBA84D183DE]:0)
[junit4] Completed on J1 in 7225.62s, 1 test, 2 errors <<< FAILURES!
[...truncated 203 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 362 suites, 1563 tests, 1 suite-level error, 1 error, 263 ignored (3 assumptions)
Total time: 165 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jrockit-jdk1.6.0_45-R28.2.7-4.1.0 -XnoOpt
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.6.0_45) - Build # 9400
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9400/
Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseSerialGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Captured an uncaught exception in thread: Thread[id=2416, name=RecoveryThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2416, name=RecoveryThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
at __randomizedtesting.SeedInfo.seed([A1E80C6C0C61B9C5:200E82747B3ED9F9]:0)
Caused by: java.lang.AssertionError: file _0_1.del does not exist
at __randomizedtesting.SeedInfo.seed([A1E80C6C0C61B9C5]:0)
at org.apache.lucene.index.IndexWriter.filesExist(IndexWriter.java:4363)
at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4429)
at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2926)
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)
at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:578)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:396)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:247)
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
deletealias the collection time out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: deletealias the collection time out:180s
at __randomizedtesting.SeedInfo.seed([A1E80C6C0C61B9C5:200E82747B3ED9F9]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
at org.apache.solr.client.solrj.request.CollectionAdminRequest.process(CollectionAdminRequest.java:271)
at org.apache.solr.client.solrj.request.CollectionAdminRequest.deleteAlias(CollectionAdminRequest.java:402)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSolrJAPICalls(CollectionsAPIDistributedZkTest.java:304)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:199)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 9604 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> 418628 T1180 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 418629 T1180 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 418632 T1180 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-CollectionsAPIDistributedZkTest-1392653916300
[junit4] 2> 418633 T1180 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 418633 T1181 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 418733 T1180 oasc.ZkTestServer.run start zk server on port:40738
[junit4] 2> 418736 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 418816 T1187 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4181c037 name:ZooKeeperConnection Watcher:127.0.0.1:40738 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 418817 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 418817 T1180 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 418827 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 418829 T1189 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f0610f5 name:ZooKeeperConnection Watcher:127.0.0.1:40738/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 418830 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 418830 T1180 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 418839 T1180 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 418846 T1180 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 418849 T1180 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 418853 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 418854 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 418859 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 418860 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 418864 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 418865 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 418869 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 418870 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 418874 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 418875 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 418878 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 418879 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 418882 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 418883 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 418888 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 418888 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 418892 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 418893 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 418897 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 418898 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 418901 T1180 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 418902 T1180 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> Using new style solr.xml
[junit4] 2> 418983 T1180 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 418986 T1180 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59348
[junit4] 2> 418986 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 418987 T1180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 418987 T1180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575
[junit4] 2> 418987 T1180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/'
[junit4] 2> 418997 T1180 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/solr.xml
[junit4] 2> 419032 T1180 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575
[junit4] 2> 419032 T1180 oasc.CoreContainer.<init> New CoreContainer 1010383210
[junit4] 2> 419032 T1180 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/]
[junit4] 2> 419034 T1180 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 419034 T1180 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 419034 T1180 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 419034 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 419035 T1180 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 419035 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 419035 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 419036 T1180 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 419036 T1180 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 419038 T1180 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 419038 T1180 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 419039 T1180 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 419039 T1180 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40738/solr
[junit4] 2> 419040 T1180 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 419040 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 419042 T1200 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50e87d09 name:ZooKeeperConnection Watcher:127.0.0.1:40738 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 419042 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 419045 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 419046 T1202 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66e207d1 name:ZooKeeperConnection Watcher:127.0.0.1:40738/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 419046 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 419048 T1180 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 419052 T1180 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 419057 T1180 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 419059 T1180 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59348_
[junit4] 2> 419060 T1180 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59348_
[junit4] 2> 419064 T1180 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 419067 T1180 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 419074 T1180 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:59348_
[junit4] 2> 419074 T1180 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 419086 T1180 oasc.Overseer.start Overseer (id=91268967058767875-127.0.0.1:59348_-n_0000000000) starting
[junit4] 2> 419091 T1180 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 419096 T1204 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 419097 T1180 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 419097 T1204 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 419101 T1180 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 419104 T1180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 419108 T1203 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 419109 T1180 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575
[junit4] 2> 419110 T1180 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/conf/
[junit4] 2> 419142 T1180 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 419142 T1180 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 419143 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 419144 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 419145 T1207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f62dd7d name:ZooKeeperConnection Watcher:127.0.0.1:40738/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 419146 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 419147 T1180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 419151 T1180 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 419222 T1180 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 419223 T1180 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 419225 T1180 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43365
[junit4] 2> 419225 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 419226 T1180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 419226 T1180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819
[junit4] 2> 419226 T1180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/'
[junit4] 2> 419236 T1180 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/solr.xml
[junit4] 2> 419269 T1180 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819
[junit4] 2> 419270 T1180 oasc.CoreContainer.<init> New CoreContainer 401880071
[junit4] 2> 419270 T1180 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/]
[junit4] 2> 419271 T1180 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 419271 T1180 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 419272 T1180 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 419272 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 419272 T1180 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 419273 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 419273 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 419273 T1180 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 419274 T1180 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 419276 T1180 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 419276 T1180 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 419276 T1180 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 419277 T1180 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40738/solr
[junit4] 2> 419277 T1180 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 419278 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 419279 T1218 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2183c3da name:ZooKeeperConnection Watcher:127.0.0.1:40738 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 419280 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 419282 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 419283 T1220 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57ff0352 name:ZooKeeperConnection Watcher:127.0.0.1:40738/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 419284 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 419290 T1180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 420294 T1180 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43365_
[junit4] 2> 420296 T1180 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43365_
[junit4] 2> 420300 T1202 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 420301 T1207 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 420301 T1220 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 420308 T1180 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819
[junit4] 2> 420310 T1180 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/conf/
[junit4] 2> 420350 T1180 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 420351 T1180 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 420351 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 420424 T1180 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 420424 T1180 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 420427 T1180 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41460
[junit4] 2> 420428 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 420428 T1180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 420428 T1180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019
[junit4] 2> 420429 T1180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/'
[junit4] 2> 420446 T1180 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/solr.xml
[junit4] 2> 420495 T1180 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019
[junit4] 2> 420496 T1180 oasc.CoreContainer.<init> New CoreContainer 620274786
[junit4] 2> 420496 T1180 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/]
[junit4] 2> 420497 T1180 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 420498 T1180 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 420498 T1180 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 420498 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 420499 T1180 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 420499 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 420499 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 420499 T1180 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 420500 T1180 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 420502 T1180 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 420502 T1180 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 420502 T1180 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 420503 T1180 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40738/solr
[junit4] 2> 420503 T1180 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 420504 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 420505 T1232 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ff030e4 name:ZooKeeperConnection Watcher:127.0.0.1:40738 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 420506 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 420508 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 420510 T1234 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e44174e name:ZooKeeperConnection Watcher:127.0.0.1:40738/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 420510 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 420517 T1180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 421524 T1180 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41460_
[junit4] 2> 421527 T1180 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41460_
[junit4] 2> 421537 T1207 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 421537 T1234 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 421538 T1220 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 421539 T1202 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 421547 T1180 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019
[junit4] 2> 421548 T1180 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/conf/
[junit4] 2> 421587 T1180 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 421587 T1180 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 421588 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 421659 T1180 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 421660 T1180 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 421671 T1180 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48095
[junit4] 2> 421672 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 421672 T1180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 421673 T1180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255
[junit4] 2> 421674 T1180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/'
[junit4] 2> 421687 T1180 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/solr.xml
[junit4] 2> 421721 T1180 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255
[junit4] 2> 421722 T1180 oasc.CoreContainer.<init> New CoreContainer 1289467625
[junit4] 2> 421722 T1180 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/]
[junit4] 2> 421723 T1180 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 421723 T1180 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 421723 T1180 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 421724 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 421724 T1180 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 421724 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 421725 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 421725 T1180 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 421725 T1180 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 421727 T1180 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 421727 T1180 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 421728 T1180 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 421728 T1180 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40738/solr
[junit4] 2> 421728 T1180 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 421729 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 421731 T1246 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6cc5a26a name:ZooKeeperConnection Watcher:127.0.0.1:40738 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 421731 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 421734 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 421735 T1248 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@206507d3 name:ZooKeeperConnection Watcher:127.0.0.1:40738/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 421735 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 421744 T1180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 422751 T1180 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48095_
[junit4] 2> 422753 T1180 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48095_
[junit4] 2> 422759 T1234 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 422759 T1207 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 422759 T1248 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 422760 T1220 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 422761 T1202 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 422766 T1180 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255
[junit4] 2> 422767 T1180 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/conf/
[junit4] 2> 422791 T1180 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 422792 T1180 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 422792 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 422864 T1180 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 422865 T1180 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 422867 T1180 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58695
[junit4] 2> 422867 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 422868 T1180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 422868 T1180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1392653920459
[junit4] 2> 422869 T1180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1392653920459/'
[junit4] 2> 422879 T1180 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1392653920459/solr.xml
[junit4] 2> 422915 T1180 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1392653920459
[junit4] 2> 422916 T1180 oasc.CoreContainer.<init> New CoreContainer 337674819
[junit4] 2> 422916 T1180 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1392653920459/]
[junit4] 2> 422917 T1180 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 422917 T1180 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 422918 T1180 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 422918 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 422918 T1180 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 422919 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 422919 T1180 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 422919 T1180 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 422920 T1180 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 422922 T1180 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 422922 T1180 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 422923 T1180 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 422923 T1180 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40738/solr
[junit4] 2> 422923 T1180 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 422924 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 422926 T1260 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62480707 name:ZooKeeperConnection Watcher:127.0.0.1:40738 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 422926 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 422929 T1180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 422931 T1262 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@363b4064 name:ZooKeeperConnection Watcher:127.0.0.1:40738/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 422931 T1180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 422940 T1180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 423945 T1180 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58695_
[junit4] 2> 423947 T1180 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58695_
[junit4] 2> 423956 T1248 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 423956 T1234 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 423957 T1202 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 423958 T1207 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 423958 T1220 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 423962 T1262 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 423967 T1180 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1392653920459
[junit4] 2> 423968 T1180 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1392653920459/conf/
[junit4] 2> 423998 T1180 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 423999 T1180 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 423999 T1180 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 424010 T1214 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&collection.configName=conf1&name=solrj_collection&replicationFactor=2&action=CREATE&router.field=myOwnField&wt=javabin&router.name=compositeId&version=2
[junit4] 2> 424012 T1202 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 424014 T1202 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 424015 T1204 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 424015 T1204 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 424015 T1204 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection
[junit4] 2> 424015 T1204 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
[junit4] 2> 424053 T1202 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 424055 T1203 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 424056 T1203 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
[junit4] 2> 424057 T1203 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
[junit4] 2> 424062 T1202 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 424065 T1234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424065 T1202 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424065 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424066 T1220 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424065 T1207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424066 T1262 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424153 T1204 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
[junit4] 2> 424154 T1204 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:41460_
[junit4] 2> 424155 T1204 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:48095_
[junit4] 2> 424158 T1204 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:43365_
[junit4] 2> 424159 T1228 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down
[junit4] 2> 424159 T1244 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down
[junit4] 2> 424159 T1204 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:59348_
[junit4] 2> 424161 T1216 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down
[junit4] 2> 424162 T1228 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 424162 T1244 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 424162 T1202 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 424163 T1216 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 424164 T1196 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down
[junit4] 2> 424166 T1202 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 424167 T1203 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 424168 T1203 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41460",
[junit4] 2> "core":"solrj_collection_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41460_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 424169 T1196 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 424175 T1203 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48095",
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:48095_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 424180 T1202 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 424185 T1203 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43365",
[junit4] 2> "core":"solrj_collection_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:43365_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 424190 T1202 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 424193 T1203 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:59348",
[junit4] 2> "core":"solrj_collection_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:59348_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 424197 T1202 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 424301 T1207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424302 T1248 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424302 T1202 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424302 T1220 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424303 T1234 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 424303 T1262 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 425162 T1228 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
[junit4] 2> 425162 T1244 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
[junit4] 2> 425164 T1216 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
[junit4] 2> 425164 T1228 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/solrj_collection_shard1_replica1
[junit4] 2> 425164 T1244 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/solrj_collection_shard1_replica2
[junit4] 2> 425165 T1244 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 425164 T1216 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/solrj_collection_shard2_replica1
[junit4] 2> 425164 T1228 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 425166 T1216 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 425167 T1216 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 425167 T1216 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 425169 T1244 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 425169 T1244 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 425170 T1196 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
[junit4] 2> 425170 T1228 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 425171 T1228 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 425171 T1196 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/solrj_collection_shard2_replica2
[junit4] 2> 425171 T1196 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 425174 T1216 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 425174 T1196 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 425174 T1216 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/solrj_collection_shard2_replica1/'
[junit4] 2> 425174 T1196 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 425178 T1228 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 425179 T1228 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/solrj_collection_shard1_replica1/'
[junit4] 2> 425180 T1244 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 425181 T1244 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/solrj_collection_shard1_replica2/'
[junit4] 2> 425181 T1196 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 425184 T1196 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/solrj_collection_shard2_replica2/'
[junit4] 2> 425284 T1244 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 425289 T1228 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 425298 T1216 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 425319 T1196 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 425398 T1196 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 425401 T1196 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 425402 T1228 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 425405 T1228 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 425412 T1196 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
[junit4] 2> 425416 T1228 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
[junit4] 2> 425419 T1244 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 425422 T1244 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 425446 T1244 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
[junit4] 2> 425458 T1216 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 425460 T1216 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 425495 T1216 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
[junit4] 2> 426531 T1228 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 426589 T1228 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 426590 T1228 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 426593 T1244 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 426594 T1244 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 426595 T1244 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 426600 T1228 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 426604 T1228 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 426606 T1244 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 426610 T1196 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 426610 T1244 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 426610 T1196 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 426611 T1196 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 426617 T1196 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 426620 T1196 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 426670 T1244 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 426673 T1216 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 426673 T1216 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 426674 T1216 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 426676 T1244 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 426678 T1228 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 426680 T1196 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 426692 T1244 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 426693 T1228 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 426694 T1216 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 426698 T1216 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 426700 T1228 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 426702 T1228 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 426703 T1244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 426703 T1244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 426703 T1196 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 426703 T1228 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 426705 T1228 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 426704 T1244 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 426708 T1244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 426708 T1244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 426709 T1244 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 426709 T1244 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/solrj_collection_shard1_replica2/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/solrj_collection_shard1_replica2/data/
[junit4] 2> 426710 T1244 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@bc10d02
[junit4] 2> 426716 T1244 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/solrj_collection_shard1_replica2/data
[junit4] 2> 426716 T1244 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/solrj_collection_shard1_replica2/data/index/
[junit4] 2> 426717 T1244 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 426717 T1244 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1392653919255/solrj_collection_shard1_replica2/data/index
[junit4] 2> 426718 T1244 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=1.8603515625, floorSegmentMB=1.1904296875, forceMergeDeletesPctAllowed=17.8928489670199, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7490457491565093
[junit4] 2> 426719 T1244 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4f5204ca lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@40131d31),segFN=segments_1,generation=1}
[junit4] 2> 426720 T1244 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 426721 T1196 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 426724 T1228 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 426725 T1228 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 426725 T1244 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 426725 T1228 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 426726 T1244 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 426726 T1228 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/solrj_collection_shard1_replica1/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/solrj_collection_shard1_replica1/data/
[junit4] 2> 426726 T1244 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 426730 T1228 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@bc10d02
[junit4] 2> 426740 T1196 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 426742 T1196 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 426742 T1196 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 426740 T1244 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 426741 T1228 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/solrj_collection_shard1_replica1/data
[junit4] 2> 426746 T1196 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 426747 T1196 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 426748 T1196 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 426748 T1196 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/solrj_collection_shard2_replica2/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/solrj_collection_shard2_replica2/data/
[junit4] 2> 426748 T1196 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@bc10d02
[junit4] 2> 426744 T1244 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 426749 T1196 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/solrj_collection_shard2_replica2/data
[junit4] 2> 426747 T1228 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/solrj_collection_shard1_replica1/data/index/
[junit4] 2> 426750 T1196 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/solrj_collection_shard2_replica2/data/index/
[junit4] 2> 426749 T1244 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 426750 T1196 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 426750 T1228 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 426751 T1196 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1392653916575/solrj_collection_shard2_replica2/data/index
[junit4] 2> 426751 T1244 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 426752 T1196 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=1.8603515625, floorSegmentMB=1.1904296875, forceMergeDeletesPctAllowed=17.8928489670199, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7490457491565093
[junit4] 2> 426752 T1228 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1392653918019/solrj_collection_shard1_replica1/data/index
[junit4] 2> 426753 T1228 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=1.8603515625, floorSegmentMB=1.1904296875, forceMergeDeletesPctAllowed=17.8928489670199, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7490457491565093
[junit4] 2> 426752 T1244 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 426753 T1196 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@5868a774 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@606f3b45),segFN=segments_1,generation=1}
[junit4] 2> 426754 T1228 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@37f13bed lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14c6731),segFN=segments_1,generation=1}
[junit4] 2> 426755 T1228 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 426754 T1244 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 426756 T1244 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 426756 T1244 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 426757 T1244 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 426757 T1244 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 426758 T1244 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 426758 T1244 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 426759 T1244 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 426754 T1196 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 426766 T1196 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 426767 T1228 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 426768 T1228 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 426768 T1228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 426769 T1228 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 426769 T1228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 426767 T1196 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 426769 T1228 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 426769 T1244 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 426770 T1228 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 426771 T1228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 426771 T1228 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 426772 T1228 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 426773 T1228 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 426773 T1228 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 426770 T1196 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 426773 T1228 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 426774 T1228 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 426774 T1196 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 426775 T1228 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 426774 T1244 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 426775 T1228 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 426775 T1196 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 426776 T1196 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 426777 T1196 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 426777 T1196 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 426778 T1196 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 426778 T1196 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 426779 T1196 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 426779 T1196 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 426779 T1196 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 426780 T1196 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 426780 T1196 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 426780 T1196 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 426776 T1244 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 426782 T1216 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 426777 T1228 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 426788 T1196 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 426791 T1216 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 426792 T1228 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 426793 T1228 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 426796 T1244 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=43, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 426797 T1228 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=43, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 426798 T1244 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4f5204ca lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@40131d31),segFN=segments_1,generation=1}
[junit4] 2> 426798 T1244 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 426799 T1244 oass.SolrIndexSearcher.<init> Opening Searcher@400ea298[solrj_collection_shard1_replica2] main
[junit4] 2> 426801 T1216 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 426802 T1268 oasc.SolrCore.registerSearcher [solrj_collection_shard1_replica2] Registered new searcher Searcher@400ea298[solrj_collection_shard1_replica2] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 426804 T1216 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 426805 T1216 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 426805 T1216 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 426793 T1196 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 426807 T1196 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 426805 T1228 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@37f13bed lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14c6731),segFN=segments_1,generation=1}
[junit4] 2> 426808 T1216 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 426805 T1244 oasc.CoreContainer.registerCore registering core: solrj_collection_shard1_replica2
[junit4] 2> 426808 T1216 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 426808 T1196 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=43, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 426810 T1196 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@5868a774 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@606f3b45),segFN=segments_1,generation=1}
[junit4] 2> 426811 T1196 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 426811 T1196 oass.SolrIndexSearcher.<init> Opening Searcher@45e24db0[solrj_collection_shard2_replica2] main
[junit4] 2> 426808 T1228 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 426809 T1216 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 426809 T1244 oasc.ZkController.register Register replica - core:solrj_collection_shard1_replica2 address:http://127.0.0.1:48095 collection:solrj_collection shard:shard1
[junit4] 2> 426815 T1216 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/solrj_collection_shard2_replica1/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/solrj_collection_shard2_replica1/data/
[junit4] 2> 426815 T1216 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@bc10d02
[junit4] 2> 426816 T1270 oasc.SolrCore.registerSearcher [solrj_collection_shard2_replica2] Registered new searcher Searcher@45e24db0[solrj_collection_shard2_replica2] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 426817 T1216 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/solrj_collection_shard2_replica1/data
[junit4] 2> 426817 T1216 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/solrj_collection_shard2_replica1/data/index/
[junit4] 2> 426814 T1228 oass.SolrIndexSearcher.<init> Opening Searcher@640676af[solrj_collection_shard1_replica1] main
[junit4] 2> 426818 T1216 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 426819 T1244 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard1/election
[junit4] 2> 426819 T1216 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1392653916819/solrj_collection_shard2_replica1/data/index
[junit4] 2> 426820 T1216 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=1.8603515625, floorSegmentMB=1.1904296875, forceMergeDeletesPctAllowed=17.8928489670199, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7490457491565093
[junit4] 2> 426821 T1216 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@52e8a6cb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@138a91bf),segFN=segments_1,generation=1}
[junit4] 2> 426821 T1216 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 426826 T1196 oasc.CoreContainer.registerCore registering core: solrj_collection_shard2_replica2
[junit4] 2> 426826 T1216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 426826 T1196 oasc.ZkController.register Register replica - core:solrj_collection_s
[...truncated too long message...]
rg.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 806386 T2403 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty11/index
[junit4] 2> 806387 T2403 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty11 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty11;done=false>>]
[junit4] 2> 806387 T2403 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty11
[junit4] 2> 806386 T1838 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 806388 T2403 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 806388 T2403 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 806386 T2097 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty10/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty10/index;done=false>>]
[junit4] 2> 806389 T2097 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty10/index
[junit4] 2> 806388 T1838 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 806389 T2097 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty10 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty10;done=false>>]
[junit4] 2> 806390 T2097 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1392654162229/jetty10
[junit4] 2> 806390 T2097 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: test params are: codec=Lucene42, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=et_EE, timezone=US/Pacific-New
[junit4] 2> NOTE: Linux 3.8.0-35-generic amd64/Sun Microsystems Inc. 1.6.0_45 (64-bit)/cpus=8,threads=1,free=123346288,total=376823808
[junit4] 2> NOTE: All tests run in this JVM: [MultiTermTest, XmlUpdateRequestHandlerTest, TestRandomMergePolicy, HighlighterConfigTest, TestSchemaResource, TestStressReorder, RequiredFieldsTest, DistributedQueryComponentCustomSortTest, SimplePostToolTest, TestOmitPositions, TestFastWriter, TestMultiCoreConfBootstrap, HdfsBasicDistributedZk2Test, DistributedSuggestComponentTest, TestFoldingMultitermQuery, TestSolrXmlPersistence, NotRequiredUniqueKeyTest, DefaultValueUpdateProcessorTest, OpenCloseCoreStressTest, TestDistribDocBasedVersion, TestUtils, TestQueryTypes, DateFieldTest, DirectUpdateHandlerTest, TestShardHandlerFactory, HdfsUnloadDistributedZkTest, CoreContainerCoreInitFailuresTest, TestDFRSimilarityFactory, TestCSVResponseWriter, TestRTGBase, DocValuesMultiTest, SpellCheckComponentTest, TestMaxScoreQueryParser, TestReloadAndDeleteDocs, TestBinaryField, TestCodecSupport, TestStressUserVersions, BadCopyFieldTest, FieldAnalysisRequestHandlerTest, TestDistributedGrouping, TestXIncludeConfig, PluginInfoTest, TestNumberUtils, TestZkChroot, IndexBasedSpellCheckerTest, TestSolr4Spatial, CurrencyFieldOpenExchangeTest, SpellingQueryConverterTest, TestFastOutputStream, TestSimpleQParserPlugin, TestComponentsName, LeaderElectionIntegrationTest, TestHighlightDedupGrouping, CursorPagingTest, TestRandomFaceting, TestSchemaNameResource, CollectionsAPIDistributedZkTest, TimeZoneUtilsTest, TestCollationFieldDocValues, TestClassNameShortening, ShardRoutingCustomTest, TermVectorComponentDistributedTest, XsltUpdateRequestHandlerTest, TestBinaryResponseWriter, SignatureUpdateProcessorFactoryTest, NumericFieldsTest, TestRecoveryHdfs, TestImplicitCoreProperties, ConnectionManagerTest, TestStressLucene, TestFieldTypeResource, DebugComponentTest, StandardRequestHandlerTest, AssignTest, TestIndexingPerformance, TestCollationField, ChangedSchemaMergeTest, TestCoreDiscovery, TestQuerySenderListener, TestUpdate, ZkNodePropsTest, ChaosMonkeySafeLeaderTest]
[junit4] Completed on J1 in 142.83s, 1 test, 1 error <<< FAILURES!
[...truncated 607 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 362 suites, 1563 tests, 2 errors, 36 ignored (8 assumptions)
Total time: 51 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.6.0_45) - Build # 9399 - Still Failing!
Posted by Mark Miller <ma...@gmail.com>.
For some reason, this call under SSL can just be super slow in tests. Rather than keep cranking the timeout, it may be better to just not run this test under SSL.
- Mark
http://about.me/markrmiller
On Feb 17, 2014, at 9:47 AM, Policeman Jenkins Server <je...@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9399/
> Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseSerialGC
>
> 1 tests failed.
> REGRESSION: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
>
> Error Message:
> Timeout occured while waiting response from server at: https://127.0.0.1:33594/collection1
>
> Stack Trace:
> org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:33594/collection1
> at __randomizedtesting.SeedInfo.seed([C680EB6F246E98D6:476665775331F8EA]:0)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:503)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
> at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
> at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
> at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.index_specific(AbstractFullDistribZkTestBase.java:698)
> at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingWithSuss(FullSolrCloudDistribCmdsTest.java:443)
> at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:134)
> at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
> at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:422)
> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:460)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:820)
> at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
> at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
> at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
> at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
> at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
> at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
> at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
> at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
> at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:395)
> ... 48 more
>
>
>
>
> Build Log:
> [...truncated 9888 lines...]
> [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
> [junit4] 2> 869903 T3600 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
> [junit4] 2> 869904 T3600 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
> [junit4] 2> 869907 T3600 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-FullSolrCloudDistribCmdsTest-1392647735836
> [junit4] 2> 869908 T3600 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4] 2> 869909 T3601 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4] 2> 870009 T3600 oasc.ZkTestServer.run start zk server on port:45482
> [junit4] 2> 870010 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 870039 T3607 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17dcbcab name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 870039 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 870040 T3600 oascc.SolrZkClient.makePath makePath: /solr
> [junit4] 2> 870046 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 870048 T3609 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b3c0918 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 870048 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 870049 T3600 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4] 2> 870053 T3600 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4] 2> 870057 T3600 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4] 2> 870061 T3600 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4] 2> 870066 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
> [junit4] 2> 870066 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4] 2> 870073 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
> [junit4] 2> 870074 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4] 2> 870079 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 870079 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
> [junit4] 2> 870083 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4] 2> 870084 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4] 2> 870089 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4] 2> 870090 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4] 2> 870098 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4] 2> 870098 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4] 2> 870104 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
> [junit4] 2> 870104 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
> [junit4] 2> 870110 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4] 2> 870111 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4] 2> 870114 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 870115 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4] 2> 870120 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4] 2> 870121 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4] 2> 870126 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4] 2> 870126 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4] 2> 870132 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 870133 T3611 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14fc530c name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 870134 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 870203 T3600 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 870206 T3600 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
> [junit4] 2> 870208 T3600 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:33594
> [junit4] 2> 870209 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 870209 T3600 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 870210 T3600 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064
> [junit4] 2> 870210 T3600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/'
> [junit4] 2> 870221 T3600 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/solr.xml
> [junit4] 2> 870258 T3600 oasc.CoreContainer.<init> New CoreContainer 1700107404
> [junit4] 2> 870258 T3600 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/]
> [junit4] 2> 870259 T3600 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 870260 T3600 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
> [junit4] 2> 870260 T3600 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 870260 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 870261 T3600 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 870261 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 870261 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 870262 T3600 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 870262 T3600 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 870266 T3600 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 870266 T3600 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 870266 T3600 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 870267 T3600 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45482/solr
> [junit4] 2> 870267 T3600 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 870268 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 870269 T3622 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@776e560c name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 870270 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 870272 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 870273 T3624 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a31c285 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 870273 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 870275 T3600 oascc.SolrZkClient.makePath makePath: /overseer/queue
> [junit4] 2> 870281 T3600 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
> [junit4] 2> 870286 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4] 2> 870289 T3600 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33594_
> [junit4] 2> 870290 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33594_
> [junit4] 2> 870295 T3600 oascc.SolrZkClient.makePath makePath: /overseer_elect
> [junit4] 2> 870298 T3600 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4] 2> 870304 T3600 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:33594_
> [junit4] 2> 870304 T3600 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4] 2> 870307 T3600 oasc.Overseer.start Overseer (id=91268562015879172-127.0.0.1:33594_-n_0000000000) starting
> [junit4] 2> 870311 T3600 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
> [junit4] 2> 870316 T3626 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4] 2> 870316 T3600 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4] 2> 870318 T3626 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
> [junit4] 2> 870322 T3600 oascc.SolrZkClient.makePath makePath: /aliases.json
> [junit4] 2> 870325 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 870329 T3625 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4] 2> 870332 T3627 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4] 2> 870332 T3627 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 870333 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 870333 T3627 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 870335 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 870336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"https://127.0.0.1:33594",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:33594_",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"control_collection",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 870336 T3625 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
> [junit4] 2> 870336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4] 2> 870339 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 870341 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 871334 T3627 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 871334 T3627 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1
> [junit4] 2> 871334 T3627 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4] 2> 871335 T3627 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 871335 T3627 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
> [junit4] 2> 871337 T3627 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 871337 T3627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1/'
> [junit4] 2> 871338 T3627 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1/lib/classes/' to classloader
> [junit4] 2> 871338 T3627 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1/lib/README' to classloader
> [junit4] 2> 871379 T3627 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
> [junit4] 2> 871416 T3627 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 871418 T3627 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 871424 T3627 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 871756 T3627 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 871758 T3627 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 871760 T3627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 871766 T3627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 871797 T3627 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 871797 T3627 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data/
> [junit4] 2> 871797 T3627 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@69c1f2c
> [junit4] 2> 871798 T3627 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data
> [junit4] 2> 871798 T3627 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data/index/
> [junit4] 2> 871798 T3627 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data/index' doesn't exist. Creating new index...
> [junit4] 2> 871799 T3627 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data/index
> [junit4] 2> 871799 T3627 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3782485278233143]
> [junit4] 2> 871800 T3627 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@62369b3b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51f81c21),segFN=segments_1,generation=1}
> [junit4] 2> 871800 T3627 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 871802 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 871802 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 871804 T3627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 871804 T3627 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 871804 T3627 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4] 2> 871805 T3627 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4] 2> 871805 T3627 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 871805 T3627 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4] 2> 871806 T3627 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4] 2> 871806 T3627 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 871806 T3627 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
> [junit4] 2> 871808 T3627 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4] 2> 871810 T3627 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4] 2> 871810 T3627 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4] 2> 871811 T3627 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=77.642578125, floorSegmentMB=1.4921875, forceMergeDeletesPctAllowed=9.41997149083999, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
> [junit4] 2> 871811 T3627 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@62369b3b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51f81c21),segFN=segments_1,generation=1}
> [junit4] 2> 871812 T3627 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 871812 T3627 oass.SolrIndexSearcher.<init> Opening Searcher@5eb32da3[collection1] main
> [junit4] 2> 871814 T3628 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5eb32da3[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 871814 T3627 oasc.CoreContainer.registerCore registering core: collection1
> [junit4] 2> 871815 T3600 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
> [junit4] 2> 871816 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4] 2> 871815 T3631 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:33594 collection:control_collection shard:shard1
> [junit4] 2> 871817 T3631 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4] 2> 871818 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 871820 T3633 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4408f970 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 871820 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 871822 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 871827 T3600 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
> [junit4] 2> 871830 T3631 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4] 2> 871833 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 871835 T3631 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4] 2> 871835 T3631 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4] 2> ASYNC NEW_CORE C2557 name=collection1 org.apache.solr.core.SolrCore@328dbe78 url=https://127.0.0.1:33594/collection1 node=127.0.0.1:33594_ C2557_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:33594, core=collection1, node_name=127.0.0.1:33594_}
> [junit4] 2> 871835 T3631 C2557 P33594 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:33594/collection1/
> [junit4] 2> 871835 T3631 C2557 P33594 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4] 2> 871836 T3631 C2557 P33594 oasc.SyncStrategy.syncToMe https://127.0.0.1:33594/collection1/ has no replicas
> [junit4] 2> 871836 T3631 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:33594/collection1/ shard1
> [junit4] 2> 871836 T3631 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4] 2> 871836 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 871842 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 871844 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 871844 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 871856 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 871859 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 871895 T3600 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
> [junit4] 2> 871895 T3600 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 871898 T3600 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
> [junit4] 2> 871899 T3600 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:48098
> [junit4] 2> 871901 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 871901 T3600 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 871901 T3600 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756
> [junit4] 2> 871902 T3600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/'
> [junit4] 2> 871912 T3600 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/solr.xml
> [junit4] 2> 871955 T3600 oasc.CoreContainer.<init> New CoreContainer 1495330379
> [junit4] 2> 871956 T3600 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/]
> [junit4] 2> 871957 T3600 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 871957 T3600 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
> [junit4] 2> 871957 T3600 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 871958 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 871958 T3600 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 871958 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 871959 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 871959 T3600 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 871960 T3600 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 871962 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 871962 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 871965 T3600 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 871965 T3600 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 871966 T3600 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 871966 T3600 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45482/solr
> [junit4] 2> 871967 T3600 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 871968 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 871969 T3644 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9ec029b name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 871969 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 871971 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 871973 T3646 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d9955c3 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 871973 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 871980 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 872001 T3631 oasc.ZkController.register We are https://127.0.0.1:33594/collection1/ and leader is https://127.0.0.1:33594/collection1/
> [junit4] 2> 872002 T3631 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:33594
> [junit4] 2> 872002 T3631 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4] 2> 872002 T3631 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4] 2> 872002 T3631 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 872004 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 872005 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 872005 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 872006 T3631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 872008 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 872008 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"https://127.0.0.1:33594",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:33594_",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"control_collection",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 872011 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 872115 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 872115 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 872115 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4] 2> 872987 T3600 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48098_
> [junit4] 2> 872989 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48098_
> [junit4] 2> 872992 T3633 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4] 2> 872993 T3624 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4] 2> 872993 T3646 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4] 2> 873000 T3647 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4] 2> 873001 T3647 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 873002 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 873002 T3647 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 873002 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 873002 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 873004 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 873005 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"https://127.0.0.1:48098",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:48098_",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 873005 T3625 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
> [junit4] 2> 873006 T3625 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4] 2> 873010 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 873112 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 873113 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 873112 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 874002 T3647 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 874003 T3647 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1
> [junit4] 2> 874003 T3647 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4] 2> 874004 T3647 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 874004 T3647 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 874005 T3647 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 874006 T3647 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1/'
> [junit4] 2> 874006 T3647 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1/lib/classes/' to classloader
> [junit4] 2> 874006 T3647 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1/lib/README' to classloader
> [junit4] 2> 874038 T3647 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
> [junit4] 2> 874074 T3647 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 874076 T3647 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 874082 T3647 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 874405 T3647 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 874407 T3647 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 874409 T3647 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 874412 T3647 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 874441 T3647 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 874441 T3647 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1/
> [junit4] 2> 874441 T3647 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@69c1f2c
> [junit4] 2> 874442 T3647 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1
> [junit4] 2> 874442 T3647 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1/index/
> [junit4] 2> 874442 T3647 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1/index' doesn't exist. Creating new index...
> [junit4] 2> 874442 T3647 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1/index
> [junit4] 2> 874443 T3647 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3782485278233143]
> [junit4] 2> 874443 T3647 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4e5b8327 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3c67312),segFN=segments_1,generation=1}
> [junit4] 2> 874444 T3647 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 874447 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 874447 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 874447 T3647 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 874448 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 874448 T3647 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4] 2> 874448 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 874448 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 874449 T3647 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 874449 T3647 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 874449 T3647 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4] 2> 874450 T3647 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4] 2> 874450 T3647 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 874450 T3647 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4] 2> 874451 T3647 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4] 2> 874451 T3647 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 874452 T3647 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
> [junit4] 2> 874454 T3647 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4] 2> 874456 T3647 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4] 2> 874457 T3647 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4] 2> 874457 T3647 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=77.642578125, floorSegmentMB=1.4921875, forceMergeDeletesPctAllowed=9.41997149083999, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
> [junit4] 2> 874458 T3647 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4e5b8327 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3c67312),segFN=segments_1,generation=1}
> [junit4] 2> 874458 T3647 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 874458 T3647 oass.SolrIndexSearcher.<init> Opening Searcher@c2cb5af[collection1] main
> [junit4] 2> 874461 T3648 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c2cb5af[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 874462 T3647 oasc.CoreContainer.registerCore registering core: collection1
> [junit4] 2> 874463 T3600 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
> [junit4] 2> 874474 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4] 2> 874473 T3651 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:48098 collection:collection1 shard:shard1
> [junit4] 2> 874476 T3651 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4] 2> 874485 T3651 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4] 2> 874488 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874489 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874489 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874490 T3651 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4] 2> 874490 T3651 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4] 2> ASYNC NEW_CORE C2558 name=collection1 org.apache.solr.core.SolrCore@72e3796b url=https://127.0.0.1:48098/collection1 node=127.0.0.1:48098_ C2558_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:48098, core=collection1, node_name=127.0.0.1:48098_}
> [junit4] 2> 874491 T3651 C2558 P48098 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:48098/collection1/
> [junit4] 2> 874491 T3651 C2558 P48098 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4] 2> 874491 T3651 C2558 P48098 oasc.SyncStrategy.syncToMe https://127.0.0.1:48098/collection1/ has no replicas
> [junit4] 2> 874491 T3651 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:48098/collection1/ shard1
> [junit4] 2> 874491 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 874492 T3651 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4] 2> 874497 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874503 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874508 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874548 T3600 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
> [junit4] 2> 874549 T3600 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 874552 T3600 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
> [junit4] 2> 874553 T3600 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:37665
> [junit4] 2> 874555 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 874555 T3600 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 874555 T3600 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404
> [junit4] 2> 874556 T3600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/'
> [junit4] 2> 874566 T3600 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/solr.xml
> [junit4] 2> 874599 T3600 oasc.CoreContainer.<init> New CoreContainer 227894861
> [junit4] 2> 874600 T3600 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/]
> [junit4] 2> 874601 T3600 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 874601 T3600 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
> [junit4] 2> 874602 T3600 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 874602 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 874602 T3600 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 874602 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 874603 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 874603 T3600 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 874603 T3600 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 874607 T3600 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 874607 T3600 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 874607 T3600 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 874608 T3600 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45482/solr
> [junit4] 2> 874608 T3600 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 874609 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 874610 T3662 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41f8a31d name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 874611 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 874612 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 874612 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 874612 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 874615 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 874617 T3664 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71933426 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 874617 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 874625 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 874655 T3651 oasc.ZkController.register We are https://127.0.0.1:48098/collection1/ and leader is https://127.0.0.1:48098/collection1/
> [junit4] 2> 874655 T3651 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:48098
> [junit4] 2> 874655 T3651 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4] 2> 874655 T3651 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4] 2> 874656 T3651 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 874657 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874657 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874657 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874658 T3651 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 874660 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 874661 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"https://127.0.0.1:48098",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:48098_",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "core_node_name":"core_node1"}
> [junit4] 2> 874665 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 874768 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 874769 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 874768 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 874769 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4] 2> 875630 T3600 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37665_
> [junit4] 2> 875631 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37665_
> [junit4] 2> 875636 T3646 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4] 2> 875637 T3664 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4] 2> 875638 T3624 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4] 2> 875638 T3633 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4] 2> 875645 T3665 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4] 2> 875645 T3665 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 875647 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 875647 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 875647 T3665 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 875647 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 875649 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 875650 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"https://127.0.0.1:37665",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:37665_",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 875650 T3625 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
> [junit4] 2> 875650 T3625 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4] 2> 875653 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 875756 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 875757 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 875757 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 875757 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 876648 T3665 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 876648 T3665 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1
> [junit4] 2> 876648 T3665 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4] 2> 876649 T3665 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 876649 T3665 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 876651 T3665 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 876651 T3665 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1/'
> [junit4] 2> 876651 T3665 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1/lib/classes/' to classloader
> [junit4] 2> 876652 T3665 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1/lib/README' to classloader
> [junit4] 2> 876684 T3665 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
> [junit4] 2> 876731 T3665 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 876734 T3665 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 876740 T3665 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4] 2> 877095 T3665 oass.IndexSchema.readSchema default search field in schema is text
> [junit4] 2> 877097 T3665 oass.IndexSchema.readSchema unique key field: id
> [junit4] 2> 877099 T3665 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 877102 T3665 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4] 2> 877130 T3665 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4] 2> 877130 T3665 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2/
> [junit4] 2> 877130 T3665 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@69c1f2c
> [junit4] 2> 877131 T3665 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2
> [junit4] 2> 877131 T3665 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2/index/
> [junit4] 2> 877131 T3665 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2/index' doesn't exist. Creating new index...
> [junit4] 2> 877132 T3665 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2/index
> [junit4] 2> 877132 T3665 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3782485278233143]
> [junit4] 2> 877133 T3665 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6e3fafc1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14569e58),segFN=segments_1,generation=1}
> [junit4] 2> 877133 T3665 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 877136 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
> [junit4] 2> 877136 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
> [junit4] 2> 877136 T3665 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4] 2> 877137 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
> [junit4] 2> 877137 T3665 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4] 2> 877137 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
> [junit4] 2> 877137 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 877138 T3665 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4] 2> 877138 T3665 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
> [junit4] 2> 877138 T3665 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4] 2> 877139 T3665 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4] 2> 877139 T3665 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4] 2> 877139 T3665 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4] 2> 877139 T3665 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4] 2> 877140 T3665 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4] 2> 877140 T3665 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
> [junit4] 2> 877142 T3665 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4] 2> 877144 T3665 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4] 2> 877144 T3665 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4] 2> 877145 T3665 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=77.642578125, floorSegmentMB=1.4921875, forceMergeDeletesPctAllowed=9.41997149083999, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
> [junit4] 2> 877145 T3665 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6e3fafc1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14569e58),segFN=segments_1,generation=1}
> [junit4] 2> 877146 T3665 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4] 2> 877146 T3665 oass.SolrIndexSearcher.<init> Opening Searcher@1f48c4cc[collection1] main
> [junit4] 2> 877148 T3666 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f48c4cc[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4] 2> 877150 T3665 oasc.CoreContainer.registerCore registering core: collection1
> [junit4] 2> 877151 T3600 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
> [junit4] 2> 877151 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4] 2> 877151 T3669 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:37665 collection:collection1 shard:shard2
> [junit4] 2> 877153 T3669 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4] 2> 877161 T3669 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
> [junit4] 2> 877164 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877164 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877164 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877165 T3669 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4] 2> 877165 T3669 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4] 2> ASYNC NEW_CORE C2559 name=collection1 org.apache.solr.core.SolrCore@246985de url=https://127.0.0.1:37665/collection1 node=127.0.0.1:37665_ C2559_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:37665, core=collection1, node_name=127.0.0.1:37665_}
> [junit4] 2> 877165 T3669 C2559 P37665 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:37665/collection1/
> [junit4] 2> 877166 T3669 C2559 P37665 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4] 2> 877166 T3669 C2559 P37665 oasc.SyncStrategy.syncToMe https://127.0.0.1:37665/collection1/ has no replicas
> [junit4] 2> 877166 T3669 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:37665/collection1/ shard2
> [junit4] 2> 877166 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 877166 T3669 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4] 2> 877173 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877177 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877182 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877222 T3600 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
> [junit4] 2> 877223 T3600 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4] 2> 877226 T3600 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
> [junit4] 2> 877226 T3600 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:57724
> [junit4] 2> 877228 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4] 2> 877229 T3600 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4] 2> 877229 T3600 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081
> [junit4] 2> 877229 T3600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/'
> [junit4] 2> 877239 T3600 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/solr.xml
> [junit4] 2> 877272 T3600 oasc.CoreContainer.<init> New CoreContainer 644080259
> [junit4] 2> 877273 T3600 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/]
> [junit4] 2> 877274 T3600 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
> [junit4] 2> 877274 T3600 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
> [junit4] 2> 877275 T3600 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4] 2> 877275 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4] 2> 877275 T3600 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4] 2> 877276 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4] 2> 877276 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4] 2> 877276 T3600 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4] 2> 877276 T3600 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4] 2> 877280 T3600 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
> [junit4] 2> 877280 T3600 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
> [junit4] 2> 877281 T3600 oasc.CoreContainer.load Host Name: 127.0.0.1
> [junit4] 2> 877281 T3600 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45482/solr
> [junit4] 2> 877281 T3600 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4] 2> 877282 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 877285 T3680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@212ba522 name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 877285 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 877298 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 877298 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 877298 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 877298 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 877301 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4] 2> 877303 T3682 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b9acc72 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 877303 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4] 2> 877312 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4] 2> 877330 T3669 oasc.ZkController.register We are https://127.0.0.1:37665/collection1/ and leader is https://127.0.0.1:37665/collection1/
> [junit4] 2> 877330 T3669 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:37665
> [junit4] 2> 877330 T3669 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4] 2> 877330 T3669 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4] 2> 877330 T3669 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 877332 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877332 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877332 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877333 T3669 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 877335 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 877336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "base_url":"https://127.0.0.1:37665",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:37665_",
> [junit4] 2> "shard":"shard2",
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "core_node_name":"core_node2"}
> [junit4] 2> 877339 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 877444 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 877444 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 877444 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 877444 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 877444 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4] 2> 878317 T3600 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57724_
> [junit4] 2> 878318 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57724_
> [junit4] 2> 878323 T3682 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 878323 T3633 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 878323 T3664 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 878325 T3646 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 878325 T3624 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4] 2> 878332 T3683 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4] 2> 878332 T3683 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4] 2> 878333 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 878333 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 878333 T3683 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4] 2> 878334 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 878335 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4] 2> 878336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"down",
> [junit4] 2> "base_url":"https://127.0.0.1:57724",
> [junit4] 2> "core":"collection1",
> [junit4] 2> "roles":null,
> [junit4] 2> "node_name":"127.0.0.1:57724_",
> [junit4] 2> "shard":null,
> [junit4] 2> "shard_range":null,
> [junit4] 2> "shard_state":"active",
> [junit4] 2> "shard_parent":null,
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "core_node_name":null}
> [junit4] 2> 878336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
> [junit4] 2> 878337 T3625 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
> [junit4] 2> 878339 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
> [junit4] 2> 878443 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 878443 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 878443 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 878443 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 878443 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4] 2> 879334 T3683 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4] 2> 879334 T3683 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/collection1
> [junit4] 2> 879334 T3683 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4] 2> 879335 T3683 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4] 2> 879336 T3683 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
> [junit4] 2> 879337 T3683 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
> [junit4] 2> 879337 T3683 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/collection1/'
> [junit4] 2> 879338 T3683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/collection1/lib/classes/' to classloader
> [junit4] 2> 879338 T3683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/collection1/lib/README' to classloader
> [junit4] 2> 879369 T3683 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
> [junit4] 2> 879405 T3683 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4] 2> 879407 T3683 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4] 2> 879412 T3683 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4]
>
> [...truncated too long message...]
>
> live nodes size: 4)
> [junit4] 2> 957756 T3719 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4] 2> 957756 T3719 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4] 2> 957756 T3719 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
> [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
> [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> [junit4] 2>
> [junit4] 2> ASYNC NEW_CORE C2605 name=collection1 org.apache.solr.core.SolrCore@6bf1b693 url=https://127.0.0.1:39388/collection1 node=127.0.0.1:39388_ C2605_STATE=coll:collection1 core:collection1 props:{state=active, base_url=https://127.0.0.1:39388, core=collection1, node_name=127.0.0.1:39388_}
> [junit4] 2> 957837 T3738 C2605 P39388 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
> [junit4] 2> 957838 T3738 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
> [junit4] 2> 957838 T3738 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@6bf1b693
> [junit4] 2> 957847 T3738 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=9,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=18,adds=18,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=30,cumulative_deletesById=1,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=4}
> [junit4] 2> 957847 T3738 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4] 2> 957848 T3738 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4] 2> 957848 T3738 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4] 2> 957850 T3738 C2605 P39388 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1fa80ce7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2f2a9a7e),segFN=segments_5,generation=5}
> [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1fa80ce7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2f2a9a7e),segFN=segments_6,generation=6}
> [junit4] 2> 957850 T3738 C2605 P39388 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
> [junit4] 2> 957852 T3738 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
> [junit4] 2> 957852 T3738 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
> [junit4] 2> 957852 T3738 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6;done=false>>]
> [junit4] 2> 957852 T3738 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6
> [junit4] 2> 957853 T3738 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6/index;done=false>>]
> [junit4] 2> 957853 T3738 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6/index
> [junit4] 2> 957853 T3738 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4] 2> 957853 T3738 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
> [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
> [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
> [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
> [junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
> [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> [junit4] 2>
> [junit4] 2> NOTE: test params are: codec=Appending, sim=DefaultSimilarity, locale=da, timezone=Pacific/Fiji
> [junit4] 2> NOTE: Linux 3.8.0-35-generic amd64/Sun Microsystems Inc. 1.6.0_45 (64-bit)/cpus=8,threads=1,free=89771616,total=234012672
> [junit4] 2> NOTE: All tests run in this JVM: [AnalysisAfterCoreReloadTest, RemoteQueryErrorTest, HdfsCollectionsAPIDistributedZkTest, AnalysisErrorHandlingTest, TestSystemIdResolver, TestLRUCache, TestMergePolicyConfig, TestFaceting, SpellPossibilityIteratorTest, UnloadDistributedZkTest, QueryElevationComponentTest, BadComponentTest, TestSolrQueryParserResource, TestRemoteStreaming, MigrateRouteKeyTest, ReturnFieldsTest, TestRandomDVFaceting, BinaryUpdateRequestHandlerTest, TestSuggestSpellingConverter, TestDocumentBuilder, TestSurroundQueryParser, AddBlockUpdateTest, TestLMJelinekMercerSimilarityFactory, SOLR749Test, TestOverriddenPrefixQueryForCustomFieldType, InfoHandlerTest, AliasIntegrationTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestFunctionQuery, IndexSchemaTest, LoggingHandlerTest, TermVectorComponentTest, TestWriterPerf, JsonLoaderTest, LukeRequestHandlerTest, TestSolrQueryParser, ZkCLITest, DistributedQueryElevationComponentTest, SolrXmlInZkTest, TestJoin, SolrIndexSplitterTest, TestQueryUtils, CircularListTest, TestCloudManagedSchemaAddField, RegexBoostProcessorTest, FastVectorHighlighterTest, ResourceLoaderTest, DocumentBuilderTest, EchoParamsTest, CoreMergeIndexesAdminHandlerTest, SpatialFilterTest, ClusterStateTest, TestBlendedInfixSuggestions, UpdateParamsTest, BlockCacheTest, SuggesterFSTTest, TestDocSet, TestValueSourceCache, QueryParsingTest, SuggesterTest, DateMathParserTest, SystemInfoHandlerTest, TestHighFrequencyDictionaryFactory, ShowFileRequestHandlerTest, TestReversedWildcardFilterFactory, TestSolrXmlPersistor, DirectUpdateHandlerOptimizeTest, PolyFieldTest, HdfsBasicDistributedZkTest, TestElisionMultitermQuery, TestExtendedDismaxParser, TestBadConfig, TermsComponentTest, TestGroupingSearch, SolrInfoMBeanTest, DistributedDebugComponentTest, MoreLikeThisHandlerTest, PeerSyncTest, CSVRequestHandlerTest, TestSchemaVersionResource, DistributedTermsComponentTest, CoreAdminHandlerTest, CustomCollectionTest, TriLevelCompositeIdRoutingTest, WordBreakSolrSpellCheckerTest, BasicDistributedZkTest, CacheHeaderTest, SearchHandlerTest, SyncSliceTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest]
> [junit4] Completed on J0 in 88.00s, 1 test, 1 error <<< FAILURES!
>
> [...truncated 530 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 362 suites, 1563 tests, 1 error, 35 ignored (7 assumptions)
>
> Total time: 47 minutes 24 seconds
> Build step 'Invoke Ant' marked build as failure
> Description set: Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseSerialGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.6.0_45) - Build # 9399
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9399/
Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:33594/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:33594/collection1
at __randomizedtesting.SeedInfo.seed([C680EB6F246E98D6:476665775331F8EA]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:503)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.index_specific(AbstractFullDistribZkTestBase.java:698)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingWithSuss(FullSolrCloudDistribCmdsTest.java:443)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:134)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:422)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:460)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:820)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:395)
... 48 more
Build Log:
[...truncated 9888 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> 869903 T3600 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 869904 T3600 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 869907 T3600 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-FullSolrCloudDistribCmdsTest-1392647735836
[junit4] 2> 869908 T3600 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 869909 T3601 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 870009 T3600 oasc.ZkTestServer.run start zk server on port:45482
[junit4] 2> 870010 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 870039 T3607 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17dcbcab name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 870039 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 870040 T3600 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 870046 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 870048 T3609 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b3c0918 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 870048 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 870049 T3600 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 870053 T3600 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 870057 T3600 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 870061 T3600 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 870066 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 870066 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 870073 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 870074 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 870079 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 870079 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 870083 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 870084 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 870089 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 870090 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 870098 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 870098 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 870104 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 870104 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 870110 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 870111 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 870114 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 870115 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 870120 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 870121 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 870126 T3600 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 870126 T3600 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 870132 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 870133 T3611 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14fc530c name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 870134 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 870203 T3600 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 870206 T3600 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
[junit4] 2> 870208 T3600 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:33594
[junit4] 2> 870209 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 870209 T3600 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 870210 T3600 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064
[junit4] 2> 870210 T3600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/'
[junit4] 2> 870221 T3600 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/solr.xml
[junit4] 2> 870258 T3600 oasc.CoreContainer.<init> New CoreContainer 1700107404
[junit4] 2> 870258 T3600 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/]
[junit4] 2> 870259 T3600 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 870260 T3600 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 870260 T3600 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 870260 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 870261 T3600 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 870261 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 870261 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 870262 T3600 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 870262 T3600 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 870266 T3600 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 870266 T3600 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 870266 T3600 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 870267 T3600 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45482/solr
[junit4] 2> 870267 T3600 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 870268 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 870269 T3622 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@776e560c name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 870270 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 870272 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 870273 T3624 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a31c285 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 870273 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 870275 T3600 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 870281 T3600 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 870286 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 870289 T3600 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33594_
[junit4] 2> 870290 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33594_
[junit4] 2> 870295 T3600 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 870298 T3600 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 870304 T3600 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:33594_
[junit4] 2> 870304 T3600 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 870307 T3600 oasc.Overseer.start Overseer (id=91268562015879172-127.0.0.1:33594_-n_0000000000) starting
[junit4] 2> 870311 T3600 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 870316 T3626 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 870316 T3600 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 870318 T3626 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 870322 T3600 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 870325 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 870329 T3625 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 870332 T3627 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 870332 T3627 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 870333 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 870333 T3627 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 870335 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 870336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33594",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33594_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 870336 T3625 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 870336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 870339 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 870341 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 871334 T3627 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 871334 T3627 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1
[junit4] 2> 871334 T3627 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 871335 T3627 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 871335 T3627 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 871337 T3627 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 871337 T3627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1/'
[junit4] 2> 871338 T3627 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1/lib/classes/' to classloader
[junit4] 2> 871338 T3627 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1/lib/README' to classloader
[junit4] 2> 871379 T3627 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 871416 T3627 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 871418 T3627 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 871424 T3627 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 871756 T3627 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 871758 T3627 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 871760 T3627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 871766 T3627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 871797 T3627 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 871797 T3627 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1392647736064/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data/
[junit4] 2> 871797 T3627 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@69c1f2c
[junit4] 2> 871798 T3627 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data
[junit4] 2> 871798 T3627 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data/index/
[junit4] 2> 871798 T3627 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 871799 T3627 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/control/data/index
[junit4] 2> 871799 T3627 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3782485278233143]
[junit4] 2> 871800 T3627 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@62369b3b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51f81c21),segFN=segments_1,generation=1}
[junit4] 2> 871800 T3627 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 871802 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 871802 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 871803 T3627 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 871804 T3627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 871804 T3627 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 871804 T3627 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 871805 T3627 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 871805 T3627 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 871805 T3627 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 871806 T3627 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 871806 T3627 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 871806 T3627 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 871808 T3627 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 871810 T3627 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 871810 T3627 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 871811 T3627 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=77.642578125, floorSegmentMB=1.4921875, forceMergeDeletesPctAllowed=9.41997149083999, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 871811 T3627 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@62369b3b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51f81c21),segFN=segments_1,generation=1}
[junit4] 2> 871812 T3627 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 871812 T3627 oass.SolrIndexSearcher.<init> Opening Searcher@5eb32da3[collection1] main
[junit4] 2> 871814 T3628 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5eb32da3[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 871814 T3627 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 871815 T3600 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4] 2> 871816 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 871815 T3631 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:33594 collection:control_collection shard:shard1
[junit4] 2> 871817 T3631 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 871818 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 871820 T3633 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4408f970 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 871820 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 871822 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 871827 T3600 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 871830 T3631 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 871833 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 871835 T3631 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 871835 T3631 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2557 name=collection1 org.apache.solr.core.SolrCore@328dbe78 url=https://127.0.0.1:33594/collection1 node=127.0.0.1:33594_ C2557_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:33594, core=collection1, node_name=127.0.0.1:33594_}
[junit4] 2> 871835 T3631 C2557 P33594 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:33594/collection1/
[junit4] 2> 871835 T3631 C2557 P33594 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 871836 T3631 C2557 P33594 oasc.SyncStrategy.syncToMe https://127.0.0.1:33594/collection1/ has no replicas
[junit4] 2> 871836 T3631 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:33594/collection1/ shard1
[junit4] 2> 871836 T3631 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 871836 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 871842 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 871844 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 871844 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 871856 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 871859 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 871895 T3600 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 871895 T3600 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 871898 T3600 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
[junit4] 2> 871899 T3600 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:48098
[junit4] 2> 871901 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 871901 T3600 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 871901 T3600 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756
[junit4] 2> 871902 T3600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/'
[junit4] 2> 871912 T3600 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/solr.xml
[junit4] 2> 871955 T3600 oasc.CoreContainer.<init> New CoreContainer 1495330379
[junit4] 2> 871956 T3600 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/]
[junit4] 2> 871957 T3600 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 871957 T3600 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 871957 T3600 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 871958 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 871958 T3600 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 871958 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 871959 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 871959 T3600 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 871960 T3600 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 871962 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 871962 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 871965 T3600 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 871965 T3600 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 871966 T3600 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 871966 T3600 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45482/solr
[junit4] 2> 871967 T3600 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 871968 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 871969 T3644 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9ec029b name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 871969 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 871971 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 871973 T3646 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d9955c3 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 871973 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 871980 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 872001 T3631 oasc.ZkController.register We are https://127.0.0.1:33594/collection1/ and leader is https://127.0.0.1:33594/collection1/
[junit4] 2> 872002 T3631 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:33594
[junit4] 2> 872002 T3631 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 872002 T3631 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 872002 T3631 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 872004 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 872005 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 872005 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 872006 T3631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 872008 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 872008 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:33594",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33594_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 872011 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 872115 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 872115 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 872115 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 872987 T3600 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48098_
[junit4] 2> 872989 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48098_
[junit4] 2> 872992 T3633 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 872993 T3624 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 872993 T3646 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 873000 T3647 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 873001 T3647 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 873002 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 873002 T3647 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 873002 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 873002 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 873004 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 873005 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:48098",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:48098_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 873005 T3625 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 873006 T3625 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 873010 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 873112 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 873113 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 873112 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 874002 T3647 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 874003 T3647 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1
[junit4] 2> 874003 T3647 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 874004 T3647 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 874004 T3647 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 874005 T3647 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 874006 T3647 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1/'
[junit4] 2> 874006 T3647 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1/lib/classes/' to classloader
[junit4] 2> 874006 T3647 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1/lib/README' to classloader
[junit4] 2> 874038 T3647 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 874074 T3647 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 874076 T3647 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 874082 T3647 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 874405 T3647 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 874407 T3647 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 874409 T3647 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 874412 T3647 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 874441 T3647 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 874441 T3647 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1392647737756/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1/
[junit4] 2> 874441 T3647 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@69c1f2c
[junit4] 2> 874442 T3647 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1
[junit4] 2> 874442 T3647 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1/index/
[junit4] 2> 874442 T3647 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 874442 T3647 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty1/index
[junit4] 2> 874443 T3647 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3782485278233143]
[junit4] 2> 874443 T3647 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4e5b8327 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3c67312),segFN=segments_1,generation=1}
[junit4] 2> 874444 T3647 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 874447 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 874447 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 874447 T3647 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 874448 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 874448 T3647 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 874448 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 874448 T3647 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 874449 T3647 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 874449 T3647 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 874449 T3647 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 874450 T3647 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 874450 T3647 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 874450 T3647 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 874451 T3647 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 874451 T3647 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 874452 T3647 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 874454 T3647 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 874456 T3647 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 874457 T3647 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 874457 T3647 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=77.642578125, floorSegmentMB=1.4921875, forceMergeDeletesPctAllowed=9.41997149083999, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 874458 T3647 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4e5b8327 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3c67312),segFN=segments_1,generation=1}
[junit4] 2> 874458 T3647 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 874458 T3647 oass.SolrIndexSearcher.<init> Opening Searcher@c2cb5af[collection1] main
[junit4] 2> 874461 T3648 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c2cb5af[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 874462 T3647 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 874463 T3600 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4] 2> 874474 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 874473 T3651 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:48098 collection:collection1 shard:shard1
[junit4] 2> 874476 T3651 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 874485 T3651 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 874488 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874489 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874489 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874490 T3651 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 874490 T3651 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2558 name=collection1 org.apache.solr.core.SolrCore@72e3796b url=https://127.0.0.1:48098/collection1 node=127.0.0.1:48098_ C2558_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:48098, core=collection1, node_name=127.0.0.1:48098_}
[junit4] 2> 874491 T3651 C2558 P48098 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:48098/collection1/
[junit4] 2> 874491 T3651 C2558 P48098 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 874491 T3651 C2558 P48098 oasc.SyncStrategy.syncToMe https://127.0.0.1:48098/collection1/ has no replicas
[junit4] 2> 874491 T3651 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:48098/collection1/ shard1
[junit4] 2> 874491 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 874492 T3651 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 874497 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874503 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874508 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874548 T3600 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 874549 T3600 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 874552 T3600 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
[junit4] 2> 874553 T3600 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:37665
[junit4] 2> 874555 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 874555 T3600 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 874555 T3600 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404
[junit4] 2> 874556 T3600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/'
[junit4] 2> 874566 T3600 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/solr.xml
[junit4] 2> 874599 T3600 oasc.CoreContainer.<init> New CoreContainer 227894861
[junit4] 2> 874600 T3600 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/]
[junit4] 2> 874601 T3600 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 874601 T3600 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 874602 T3600 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 874602 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 874602 T3600 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 874602 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 874603 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 874603 T3600 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 874603 T3600 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 874607 T3600 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 874607 T3600 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 874607 T3600 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 874608 T3600 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45482/solr
[junit4] 2> 874608 T3600 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 874609 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 874610 T3662 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41f8a31d name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 874611 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 874612 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 874612 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 874612 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 874615 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 874617 T3664 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71933426 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 874617 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 874625 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 874655 T3651 oasc.ZkController.register We are https://127.0.0.1:48098/collection1/ and leader is https://127.0.0.1:48098/collection1/
[junit4] 2> 874655 T3651 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:48098
[junit4] 2> 874655 T3651 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 874655 T3651 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 874656 T3651 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 874657 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874657 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874657 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874658 T3651 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 874660 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 874661 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:48098",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:48098_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 874665 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 874768 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 874769 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 874768 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 874769 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 875630 T3600 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37665_
[junit4] 2> 875631 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37665_
[junit4] 2> 875636 T3646 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 875637 T3664 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 875638 T3624 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 875638 T3633 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 875645 T3665 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 875645 T3665 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 875647 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 875647 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 875647 T3665 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 875647 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 875649 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 875650 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37665",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37665_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 875650 T3625 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 875650 T3625 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 875653 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 875756 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 875757 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 875757 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 875757 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 876648 T3665 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 876648 T3665 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1
[junit4] 2> 876648 T3665 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 876649 T3665 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 876649 T3665 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 876651 T3665 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 876651 T3665 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1/'
[junit4] 2> 876651 T3665 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1/lib/classes/' to classloader
[junit4] 2> 876652 T3665 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1/lib/README' to classloader
[junit4] 2> 876684 T3665 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 876731 T3665 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 876734 T3665 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 876740 T3665 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 877095 T3665 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 877097 T3665 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 877099 T3665 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 877102 T3665 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 877130 T3665 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 877130 T3665 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1392647740404/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2/
[junit4] 2> 877130 T3665 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@69c1f2c
[junit4] 2> 877131 T3665 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2
[junit4] 2> 877131 T3665 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2/index/
[junit4] 2> 877131 T3665 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 877132 T3665 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty2/index
[junit4] 2> 877132 T3665 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3782485278233143]
[junit4] 2> 877133 T3665 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6e3fafc1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14569e58),segFN=segments_1,generation=1}
[junit4] 2> 877133 T3665 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 877136 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 877136 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 877136 T3665 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 877137 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 877137 T3665 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 877137 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 877137 T3665 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 877138 T3665 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 877138 T3665 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 877138 T3665 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 877139 T3665 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 877139 T3665 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 877139 T3665 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 877139 T3665 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 877140 T3665 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 877140 T3665 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 877142 T3665 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 877144 T3665 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 877144 T3665 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 877145 T3665 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=77.642578125, floorSegmentMB=1.4921875, forceMergeDeletesPctAllowed=9.41997149083999, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 877145 T3665 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6e3fafc1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14569e58),segFN=segments_1,generation=1}
[junit4] 2> 877146 T3665 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 877146 T3665 oass.SolrIndexSearcher.<init> Opening Searcher@1f48c4cc[collection1] main
[junit4] 2> 877148 T3666 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f48c4cc[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 877150 T3665 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 877151 T3600 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4] 2> 877151 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 877151 T3669 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:37665 collection:collection1 shard:shard2
[junit4] 2> 877153 T3669 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 877161 T3669 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 877164 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877164 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877164 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877165 T3669 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 877165 T3669 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2559 name=collection1 org.apache.solr.core.SolrCore@246985de url=https://127.0.0.1:37665/collection1 node=127.0.0.1:37665_ C2559_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:37665, core=collection1, node_name=127.0.0.1:37665_}
[junit4] 2> 877165 T3669 C2559 P37665 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:37665/collection1/
[junit4] 2> 877166 T3669 C2559 P37665 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 877166 T3669 C2559 P37665 oasc.SyncStrategy.syncToMe https://127.0.0.1:37665/collection1/ has no replicas
[junit4] 2> 877166 T3669 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:37665/collection1/ shard2
[junit4] 2> 877166 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 877166 T3669 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 877173 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877177 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877182 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877222 T3600 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 877223 T3600 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 877226 T3600 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
[junit4] 2> 877226 T3600 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:57724
[junit4] 2> 877228 T3600 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 877229 T3600 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 877229 T3600 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081
[junit4] 2> 877229 T3600 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/'
[junit4] 2> 877239 T3600 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/solr.xml
[junit4] 2> 877272 T3600 oasc.CoreContainer.<init> New CoreContainer 644080259
[junit4] 2> 877273 T3600 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/]
[junit4] 2> 877274 T3600 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 877274 T3600 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 877275 T3600 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 877275 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 877275 T3600 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 877276 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 877276 T3600 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 877276 T3600 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 877276 T3600 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 877280 T3600 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 877280 T3600 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 877281 T3600 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 877281 T3600 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45482/solr
[junit4] 2> 877281 T3600 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 877282 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 877285 T3680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@212ba522 name:ZooKeeperConnection Watcher:127.0.0.1:45482 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 877285 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 877298 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 877298 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 877298 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 877298 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 877301 T3600 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 877303 T3682 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b9acc72 name:ZooKeeperConnection Watcher:127.0.0.1:45482/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 877303 T3600 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 877312 T3600 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 877330 T3669 oasc.ZkController.register We are https://127.0.0.1:37665/collection1/ and leader is https://127.0.0.1:37665/collection1/
[junit4] 2> 877330 T3669 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:37665
[junit4] 2> 877330 T3669 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 877330 T3669 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 877330 T3669 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 877332 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877332 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877332 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877333 T3669 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 877335 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 877336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:37665",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37665_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 877339 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 877444 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 877444 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 877444 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 877444 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 877444 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 878317 T3600 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57724_
[junit4] 2> 878318 T3600 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57724_
[junit4] 2> 878323 T3682 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 878323 T3633 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 878323 T3664 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 878325 T3646 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 878325 T3624 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 878332 T3683 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 878332 T3683 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 878333 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 878333 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 878333 T3683 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 878334 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 878335 T3625 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 878336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:57724",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:57724_",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 878336 T3625 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 878337 T3625 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 878339 T3624 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 878443 T3624 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 878443 T3633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 878443 T3646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 878443 T3682 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 878443 T3664 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 879334 T3683 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 879334 T3683 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/collection1
[junit4] 2> 879334 T3683 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 879335 T3683 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 879336 T3683 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 879337 T3683 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 879337 T3683 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/collection1/'
[junit4] 2> 879338 T3683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/collection1/lib/classes/' to classloader
[junit4] 2> 879338 T3683 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1392647743081/collection1/lib/README' to classloader
[junit4] 2> 879369 T3683 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 879405 T3683 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 879407 T3683 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 879412 T3683 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4]
[...truncated too long message...]
live nodes size: 4)
[junit4] 2> 957756 T3719 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 957756 T3719 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 957756 T3719 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> ASYNC NEW_CORE C2605 name=collection1 org.apache.solr.core.SolrCore@6bf1b693 url=https://127.0.0.1:39388/collection1 node=127.0.0.1:39388_ C2605_STATE=coll:collection1 core:collection1 props:{state=active, base_url=https://127.0.0.1:39388, core=collection1, node_name=127.0.0.1:39388_}
[junit4] 2> 957837 T3738 C2605 P39388 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
[junit4] 2> 957838 T3738 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4] 2> 957838 T3738 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@6bf1b693
[junit4] 2> 957847 T3738 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=9,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=18,adds=18,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=30,cumulative_deletesById=1,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=4}
[junit4] 2> 957847 T3738 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 957848 T3738 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 957848 T3738 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 957850 T3738 C2605 P39388 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1fa80ce7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2f2a9a7e),segFN=segments_5,generation=5}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1fa80ce7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2f2a9a7e),segFN=segments_6,generation=6}
[junit4] 2> 957850 T3738 C2605 P39388 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4] 2> 957852 T3738 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 957852 T3738 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 957852 T3738 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6;done=false>>]
[junit4] 2> 957852 T3738 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6
[junit4] 2> 957853 T3738 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6/index;done=false>>]
[junit4] 2> 957853 T3738 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1392647735836/jetty6/index
[junit4] 2> 957853 T3738 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 957853 T3738 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Appending, sim=DefaultSimilarity, locale=da, timezone=Pacific/Fiji
[junit4] 2> NOTE: Linux 3.8.0-35-generic amd64/Sun Microsystems Inc. 1.6.0_45 (64-bit)/cpus=8,threads=1,free=89771616,total=234012672
[junit4] 2> NOTE: All tests run in this JVM: [AnalysisAfterCoreReloadTest, RemoteQueryErrorTest, HdfsCollectionsAPIDistributedZkTest, AnalysisErrorHandlingTest, TestSystemIdResolver, TestLRUCache, TestMergePolicyConfig, TestFaceting, SpellPossibilityIteratorTest, UnloadDistributedZkTest, QueryElevationComponentTest, BadComponentTest, TestSolrQueryParserResource, TestRemoteStreaming, MigrateRouteKeyTest, ReturnFieldsTest, TestRandomDVFaceting, BinaryUpdateRequestHandlerTest, TestSuggestSpellingConverter, TestDocumentBuilder, TestSurroundQueryParser, AddBlockUpdateTest, TestLMJelinekMercerSimilarityFactory, SOLR749Test, TestOverriddenPrefixQueryForCustomFieldType, InfoHandlerTest, AliasIntegrationTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestFunctionQuery, IndexSchemaTest, LoggingHandlerTest, TermVectorComponentTest, TestWriterPerf, JsonLoaderTest, LukeRequestHandlerTest, TestSolrQueryParser, ZkCLITest, DistributedQueryElevationComponentTest, SolrXmlInZkTest, TestJoin, SolrIndexSplitterTest, TestQueryUtils, CircularListTest, TestCloudManagedSchemaAddField, RegexBoostProcessorTest, FastVectorHighlighterTest, ResourceLoaderTest, DocumentBuilderTest, EchoParamsTest, CoreMergeIndexesAdminHandlerTest, SpatialFilterTest, ClusterStateTest, TestBlendedInfixSuggestions, UpdateParamsTest, BlockCacheTest, SuggesterFSTTest, TestDocSet, TestValueSourceCache, QueryParsingTest, SuggesterTest, DateMathParserTest, SystemInfoHandlerTest, TestHighFrequencyDictionaryFactory, ShowFileRequestHandlerTest, TestReversedWildcardFilterFactory, TestSolrXmlPersistor, DirectUpdateHandlerOptimizeTest, PolyFieldTest, HdfsBasicDistributedZkTest, TestElisionMultitermQuery, TestExtendedDismaxParser, TestBadConfig, TermsComponentTest, TestGroupingSearch, SolrInfoMBeanTest, DistributedDebugComponentTest, MoreLikeThisHandlerTest, PeerSyncTest, CSVRequestHandlerTest, TestSchemaVersionResource, DistributedTermsComponentTest, CoreAdminHandlerTest, CustomCollectionTest, TriLevelCompositeIdRoutingTest, WordBreakSolrSpellCheckerTest, BasicDistributedZkTest, CacheHeaderTest, SearchHandlerTest, SyncSliceTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest]
[junit4] Completed on J0 in 88.00s, 1 test, 1 error <<< FAILURES!
[...truncated 530 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 362 suites, 1563 tests, 1 error, 35 ignored (7 assumptions)
Total time: 47 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0-fcs-b129) - Build #
9398 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9398/
Java: 64bit/jdk1.8.0-fcs-b129 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.testDistribSearch
Error Message:
document count mismatch. control=163 sum(shards)=164 cloudClient=164
Stack Trace:
java.lang.AssertionError: document count mismatch. control=163 sum(shards)=164 cloudClient=164
at __randomizedtesting.SeedInfo.seed([91F782F333BE8A65:10110CEB44E1EA59]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1230)
at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:208)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
REGRESSION: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Wrong doc count on shard1_0 expected:<274> but was:<275>
Stack Trace:
java.lang.AssertionError: Wrong doc count on shard1_0 expected:<274> but was:<275>
at __randomizedtesting.SeedInfo.seed([91F782F333BE8A65:10110CEB44E1EA59]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.ShardSplitTest.checkDocCountsAndShardStates(ShardSplitTest.java:477)
at org.apache.solr.cloud.ShardSplitTest.splitByUniqueKeyTest(ShardSplitTest.java:249)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:113)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Build Log:
[...truncated 10142 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> 24061 T191 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 24061 T191 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /m/t
[junit4] 2> 24068 T191 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-ShardSplitTest-1392640359135
[junit4] 2> 24071 T191 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 24076 T192 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 24174 T191 oasc.ZkTestServer.run start zk server on port:37329
[junit4] 2> 24281 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 24312 T193 oazs.NIOServerCnxn.doIO WARN Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4] 2> 25910 T198 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43bc9d4d name:ZooKeeperConnection Watcher:127.0.0.1:37329 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 25910 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 25912 T191 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 25950 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 25954 T200 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35554140 name:ZooKeeperConnection Watcher:127.0.0.1:37329/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 25955 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 25964 T191 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 25978 T191 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 25984 T191 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 25988 T191 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 25996 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 25997 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 26008 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 26009 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 26024 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 26025 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 26031 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 26033 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 26049 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 26050 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 26056 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 26057 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 26065 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 26067 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 26075 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 26076 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 26081 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 26082 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 26087 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 26087 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 26092 T191 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 26092 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 26101 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 26104 T202 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68d82b88 name:ZooKeeperConnection Watcher:127.0.0.1:37329/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 26104 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 26227 T191 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 26236 T191 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 26247 T191 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:37242
[junit4] 2> 26252 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 26252 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 26253 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ShardSplitTest-controljetty-1392640361178
[junit4] 2> 26253 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-controljetty-1392640361178/'
[junit4] 2> 26265 T191 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-controljetty-1392640361178/solr.xml
[junit4] 2> 26279 T191 oasc.CoreContainer.<init> New CoreContainer 1370104773
[junit4] 2> 26280 T191 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ShardSplitTest-controljetty-1392640361178/]
[junit4] 2> 26281 T191 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 26281 T191 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 26281 T191 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 26282 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 26282 T191 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 26282 T191 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 26282 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 26282 T191 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 26283 T191 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 26286 T191 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 26287 T191 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 26288 T191 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 26289 T191 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37329/solr
[junit4] 2> 26300 T191 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 26302 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 26306 T213 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e27b493 name:ZooKeeperConnection Watcher:127.0.0.1:37329 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 26307 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 26313 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 26316 T215 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66f54fba name:ZooKeeperConnection Watcher:127.0.0.1:37329/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 26316 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 26323 T191 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 26334 T191 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 26345 T191 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 26348 T191 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37242_m%2Ft
[junit4] 2> 26354 T191 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37242_m%2Ft
[junit4] 2> 26360 T191 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 26363 T191 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 26374 T191 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:37242_m%2Ft
[junit4] 2> 26375 T191 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 26377 T191 oasc.Overseer.start Overseer (id=91268078585905156-127.0.0.1:37242_m%2Ft-n_0000000000) starting
[junit4] 2> 26392 T191 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 26413 T217 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 26413 T191 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 26420 T217 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 26425 T191 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 26434 T191 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 26446 T216 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 26465 T218 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 26466 T218 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 26471 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 26471 T218 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 26478 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 26479 T216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37242/m/t",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37242_m%2Ft",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 26479 T216 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 26491 T216 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 26499 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 26510 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 27472 T218 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 27473 T218 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ShardSplitTest-controljetty-1392640361178/collection1
[junit4] 2> 27473 T218 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 27474 T218 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 27474 T218 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 27475 T218 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 27476 T218 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-controljetty-1392640361178/collection1/'
[junit4] 2> 27476 T218 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1392640361178/collection1/lib/classes/' to classloader
[junit4] 2> 27477 T218 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-controljetty-1392640361178/collection1/lib/README' to classloader
[junit4] 2> 27516 T218 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 27527 T218 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 27532 T218 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 27538 T218 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 27772 T218 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 27774 T218 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 27776 T218 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 27780 T218 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 27835 T218 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 27836 T218 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ShardSplitTest-controljetty-1392640361178/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1392640359135/control/data/
[junit4] 2> 27836 T218 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@33f04ca1
[junit4] 2> 27842 T218 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1392640359135/control/data
[junit4] 2> 27842 T218 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1392640359135/control/data/index/
[junit4] 2> 27843 T218 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1392640359135/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 27843 T218 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1392640359135/control/data/index
[junit4] 2> 27844 T218 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.38562916967884353]
[junit4] 2> 27845 T218 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6c9018eb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3ee88c57),segFN=segments_1,generation=1}
[junit4] 2> 27845 T218 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 27857 T218 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 27857 T218 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 27858 T218 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 27858 T218 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 27862 T218 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 27862 T218 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 27862 T218 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 27863 T218 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 27863 T218 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 27866 T218 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 27868 T218 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 27868 T218 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 27868 T218 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 27871 T218 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 27871 T218 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 27872 T218 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 27875 T218 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 27878 T218 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 27878 T218 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 27879 T218 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=99.376953125, floorSegmentMB=1.8330078125, forceMergeDeletesPctAllowed=23.57143174271409, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 27881 T218 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6c9018eb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3ee88c57),segFN=segments_1,generation=1}
[junit4] 2> 27881 T218 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 27881 T218 oass.SolrIndexSearcher.<init> Opening Searcher@2e4c2e25[collection1] main
[junit4] 2> 27906 T219 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2e4c2e25[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 27910 T218 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 27914 T191 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 27915 T222 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:37242/m/t collection:control_collection shard:shard1
[junit4] 2> 27915 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 27926 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 27931 T224 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@431f1ac8 name:ZooKeeperConnection Watcher:127.0.0.1:37329/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 27931 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 27933 T191 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 27935 T222 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 27939 T191 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 27962 T222 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 27973 T222 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 27974 T222 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C27 name=collection1 org.apache.solr.core.SolrCore@82a7983 url=https://127.0.0.1:37242/m/t/collection1 node=127.0.0.1:37242_m%2Ft C27_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:37242/m/t, core=collection1, node_name=127.0.0.1:37242_m%2Ft}
[junit4] 2> 27974 T222 C27 P37242 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:37242/m/t/collection1/
[junit4] 2> 27975 T222 C27 P37242 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 27975 T222 C27 P37242 oasc.SyncStrategy.syncToMe https://127.0.0.1:37242/m/t/collection1/ has no replicas
[junit4] 2> 27975 T222 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:37242/m/t/collection1/ shard1
[junit4] 2> 27975 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 27975 T222 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 27977 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 27989 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 27995 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 28000 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 28001 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 28003 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 28046 T222 oasc.ZkController.register We are https://127.0.0.1:37242/m/t/collection1/ and leader is https://127.0.0.1:37242/m/t/collection1/
[junit4] 2> 28046 T222 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:37242/m/t
[junit4] 2> 28046 T222 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 28046 T222 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 28046 T222 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 28051 T222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 28051 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 28055 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 28056 T216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:37242/m/t",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37242_m%2Ft",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 28060 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 28092 T191 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 28093 T191 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 28104 T191 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 28106 T191 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:38615
[junit4] 2> 28107 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 28108 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 28108 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ShardSplitTest-jetty1-1392640363006
[junit4] 2> 28108 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty1-1392640363006/'
[junit4] 2> 28123 T191 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty1-1392640363006/solr.xml
[junit4] 2> 28131 T191 oasc.CoreContainer.<init> New CoreContainer 798440378
[junit4] 2> 28132 T191 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ShardSplitTest-jetty1-1392640363006/]
[junit4] 2> 28132 T191 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 28132 T191 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 28133 T191 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 28133 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 28133 T191 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 28133 T191 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 28134 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 28134 T191 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 28134 T191 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 28138 T191 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 28139 T191 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 28139 T191 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 28139 T191 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37329/solr
[junit4] 2> 28141 T191 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 28141 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 28145 T235 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5de75a5f name:ZooKeeperConnection Watcher:127.0.0.1:37329 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 28146 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 28151 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 28157 T237 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37c78246 name:ZooKeeperConnection Watcher:127.0.0.1:37329/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 28158 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 28163 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 28163 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 28167 T191 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 29172 T191 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38615_m%2Ft
[junit4] 2> 29174 T191 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38615_m%2Ft
[junit4] 2> 29177 T224 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 29177 T215 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 29178 T237 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 29186 T238 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 29186 T238 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 29189 T238 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 29190 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 29190 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 29190 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 29192 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 29193 T216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38615/m/t",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38615_m%2Ft",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 29194 T216 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 29194 T216 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 29198 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 29302 T237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 29302 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 29302 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 30190 T238 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 30190 T238 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ShardSplitTest-jetty1-1392640363006/collection1
[junit4] 2> 30190 T238 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 30191 T238 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 30191 T238 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 30192 T238 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 30192 T238 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty1-1392640363006/collection1/'
[junit4] 2> 30193 T238 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1392640363006/collection1/lib/classes/' to classloader
[junit4] 2> 30193 T238 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty1-1392640363006/collection1/lib/README' to classloader
[junit4] 2> 30217 T238 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 30232 T238 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 30233 T238 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 30240 T238 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 30374 T238 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 30376 T238 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 30377 T238 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 30381 T238 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 30424 T238 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 30424 T238 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ShardSplitTest-jetty1-1392640363006/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty1/
[junit4] 2> 30424 T238 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@33f04ca1
[junit4] 2> 30425 T238 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty1
[junit4] 2> 30426 T238 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty1/index/
[junit4] 2> 30426 T238 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 30426 T238 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty1/index
[junit4] 2> 30427 T238 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.38562916967884353]
[junit4] 2> 30428 T238 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@667845a1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1c3ce370),segFN=segments_1,generation=1}
[junit4] 2> 30428 T238 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 30435 T238 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 30436 T238 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 30436 T238 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 30436 T238 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 30436 T238 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 30437 T238 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 30437 T238 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 30437 T238 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 30437 T238 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 30439 T238 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 30440 T238 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 30440 T238 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 30441 T238 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 30442 T238 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 30442 T238 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 30444 T238 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 30446 T238 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 30448 T238 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 30448 T238 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 30450 T238 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=99.376953125, floorSegmentMB=1.8330078125, forceMergeDeletesPctAllowed=23.57143174271409, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 30451 T238 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@667845a1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1c3ce370),segFN=segments_1,generation=1}
[junit4] 2> 30451 T238 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 30452 T238 oass.SolrIndexSearcher.<init> Opening Searcher@3eafc799[collection1] main
[junit4] 2> 30459 T239 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3eafc799[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 30464 T238 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 30465 T191 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 30465 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 30469 T242 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:38615/m/t collection:collection1 shard:shard2
[junit4] 2> 30471 T242 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 30485 T242 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 30489 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30489 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30489 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30489 T242 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 30490 T242 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 30491 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C28 name=collection1 org.apache.solr.core.SolrCore@c47392f url=https://127.0.0.1:38615/m/t/collection1 node=127.0.0.1:38615_m%2Ft C28_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:38615/m/t, core=collection1, node_name=127.0.0.1:38615_m%2Ft}
[junit4] 2> 30490 T242 C28 P38615 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:38615/m/t/collection1/
[junit4] 2> 30506 T242 C28 P38615 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 30506 T242 C28 P38615 oasc.SyncStrategy.syncToMe https://127.0.0.1:38615/m/t/collection1/ has no replicas
[junit4] 2> 30506 T242 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:38615/m/t/collection1/ shard2
[junit4] 2> 30506 T242 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 30507 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30513 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30517 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30576 T191 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 30577 T191 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 30582 T191 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 30584 T191 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:56689
[junit4] 2> 30586 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 30587 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 30587 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ShardSplitTest-jetty2-1392640365534
[junit4] 2> 30588 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty2-1392640365534/'
[junit4] 2> 30599 T191 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty2-1392640365534/solr.xml
[junit4] 2> 30611 T191 oasc.CoreContainer.<init> New CoreContainer 535357013
[junit4] 2> 30612 T191 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ShardSplitTest-jetty2-1392640365534/]
[junit4] 2> 30612 T191 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 30612 T191 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 30613 T191 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 30613 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 30613 T191 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 30614 T191 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 30614 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 30614 T191 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 30615 T191 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 30620 T191 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 30621 T191 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 30621 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 30621 T237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 30621 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 30621 T191 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 30625 T191 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37329/solr
[junit4] 2> 30627 T191 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 30629 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 30633 T253 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a09896f name:ZooKeeperConnection Watcher:127.0.0.1:37329 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 30633 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 30637 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 30638 T255 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6fb354a8 name:ZooKeeperConnection Watcher:127.0.0.1:37329/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 30639 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 30643 T191 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 30664 T242 oasc.ZkController.register We are https://127.0.0.1:38615/m/t/collection1/ and leader is https://127.0.0.1:38615/m/t/collection1/
[junit4] 2> 30664 T242 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:38615/m/t
[junit4] 2> 30665 T242 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 30665 T242 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 30665 T242 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 30666 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30666 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30667 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30667 T242 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 30668 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 30669 T216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:38615/m/t",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38615_m%2Ft",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 30672 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 30774 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 30774 T237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 30774 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 30774 T255 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 31646 T191 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56689_m%2Ft
[junit4] 2> 31647 T191 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56689_m%2Ft
[junit4] 2> 31649 T255 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 31649 T215 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 31650 T224 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 31650 T237 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 31654 T256 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 31654 T256 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 31655 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 31656 T256 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 31656 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 31656 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 31657 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 31658 T216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:56689/m/t",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:56689_m%2Ft",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 31658 T216 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 31658 T216 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 31661 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 31763 T255 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 31763 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 31763 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 31763 T237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 32656 T256 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 32656 T256 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ShardSplitTest-jetty2-1392640365534/collection1
[junit4] 2> 32656 T256 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 32657 T256 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 32657 T256 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 32658 T256 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 32658 T256 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty2-1392640365534/collection1/'
[junit4] 2> 32659 T256 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1392640365534/collection1/lib/classes/' to classloader
[junit4] 2> 32659 T256 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty2-1392640365534/collection1/lib/README' to classloader
[junit4] 2> 32674 T256 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 32681 T256 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 32683 T256 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 32686 T256 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 32769 T256 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 32770 T256 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 32771 T256 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 32774 T256 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 32792 T256 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 32792 T256 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ShardSplitTest-jetty2-1392640365534/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty2/
[junit4] 2> 32792 T256 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@33f04ca1
[junit4] 2> 32794 T256 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty2
[junit4] 2> 32794 T256 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty2/index/
[junit4] 2> 32794 T256 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 32794 T256 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1392640359135/jetty2/index
[junit4] 2> 32795 T256 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.38562916967884353]
[junit4] 2> 32795 T256 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1b97a488 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@151aec),segFN=segments_1,generation=1}
[junit4] 2> 32795 T256 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 32801 T256 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 32801 T256 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 32801 T256 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 32801 T256 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 32802 T256 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 32802 T256 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 32802 T256 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 32802 T256 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 32802 T256 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 32803 T256 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 32804 T256 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 32804 T256 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 32804 T256 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 32805 T256 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 32805 T256 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 32806 T256 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 32807 T256 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 32809 T256 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 32809 T256 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 32810 T256 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=99.376953125, floorSegmentMB=1.8330078125, forceMergeDeletesPctAllowed=23.57143174271409, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 32810 T256 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1b97a488 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@151aec),segFN=segments_1,generation=1}
[junit4] 2> 32811 T256 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 32811 T256 oass.SolrIndexSearcher.<init> Opening Searcher@1503bee7[collection1] main
[junit4] 2> 32815 T257 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1503bee7[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 32819 T256 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 32820 T191 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 32820 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 32820 T260 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:56689/m/t collection:collection1 shard:shard1
[junit4] 2> 32825 T260 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 32834 T260 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 32836 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32836 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32836 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32837 T260 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 32837 T260 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C29 name=collection1 org.apache.solr.core.SolrCore@6bfe6412 url=https://127.0.0.1:56689/m/t/collection1 node=127.0.0.1:56689_m%2Ft C29_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:56689/m/t, core=collection1, node_name=127.0.0.1:56689_m%2Ft}
[junit4] 2> 32837 T260 C29 P56689 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:56689/m/t/collection1/
[junit4] 2> 32838 T260 C29 P56689 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 32838 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 32838 T260 C29 P56689 oasc.SyncStrategy.syncToMe https://127.0.0.1:56689/m/t/collection1/ has no replicas
[junit4] 2> 32838 T260 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:56689/m/t/collection1/ shard1
[junit4] 2> 32838 T260 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 32841 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32844 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32854 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32908 T191 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 32909 T191 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 32912 T191 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 32914 T191 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:59709
[junit4] 2> 32916 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 32917 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 32917 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ShardSplitTest-jetty3-1392640367889
[junit4] 2> 32918 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty3-1392640367889/'
[junit4] 2> 32927 T191 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-jetty3-1392640367889/solr.xml
[junit4] 2> 32934 T191 oasc.CoreContainer.<init> New CoreContainer 1049502267
[junit4] 2> 32934 T191 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ShardSplitTest-jetty3-1392640367889/]
[junit4] 2> 32938 T191 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 32939 T191 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 32939 T191 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 32939 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 32939 T191 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 32940 T191 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 32940 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 32940 T191 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 32940 T191 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 32943 T191 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 32944 T191 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 32944 T191 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 32944 T191 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37329/solr
[junit4] 2> 32944 T191 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 32945 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 32948 T271 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@99e6b95 name:ZooKeeperConnection Watcher:127.0.0.1:37329 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 32948 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 32951 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 32952 T273 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3722eadc name:ZooKeeperConnection Watcher:127.0.0.1:37329/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 32952 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 32957 T255 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 32957 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 32957 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 32957 T237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 32959 T191 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 32995 T260 oasc.ZkController.register We are https://127.0.0.1:56689/m/t/collection1/ and leader is https://127.0.0.1:56689/m/t/collection1/
[junit4] 2> 32995 T260 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:56689/m/t
[junit4] 2> 32995 T260 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 32995 T260 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 32995 T260 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 32997 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32997 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32997 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 32997 T260 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 32998 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 32999 T216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:56689/m/t",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:56689_m%2Ft",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 33001 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 33103 T255 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 33103 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 33103 T237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 33103 T273 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 33103 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 33962 T191 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59709_m%2Ft
[junit4] 2> 33963 T191 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59709_m%2Ft
[junit4] 2> 33966 T273 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 33966 T237 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 33966 T255 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 33966 T215 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 33966 T224 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 33970 T274 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 33970 T274 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 33971 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 33971 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 33971 T274 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 33971 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 33973 T216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 33973 T216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:59709/m/t",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:59709_m%2Ft",
[junit4] 2> "shard":null,
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 33974 T216 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 33974 T216 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 33976 T215 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 34078 T255 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 34078 T237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 34078 T215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 34078 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 34078 T273 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 34972 T274 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 34972 T274 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ShardSplitTest-jetty3-1392640367889/collection1
[junit4] 2> 34972 T274 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 34973 T274 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 34973 T274 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 34974 T274 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 34974 T274 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ShardSplitTest-jetty3-1392640367889/collection1/'
[junit4] 2> 34975 T274 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1392640367889/collection1/lib/classes/' to classloader
[junit4] 2> 34975 T274 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ShardSplitTest-jetty3-1392640367889/collection1/lib/README' to classloader
[junit4] 2> 34990 T274 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
[junit4] 2> 34999 T274 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 35000 T274 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 35004 T274 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 35091 T274 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 35092 T274 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 35093 T274 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xm
[...truncated too long message...]
)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 1006517 T871 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1392641283451/jetty4 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1392641283451/jetty4;done=false>>]
[junit4] 2> 1006519 T871 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1392641283451/jetty4
[junit4] 2> 1006517 T931 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 1006519 T871 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1006519 T871 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4] 2> 1006520 T871 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1006520 T871 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 1006521 T931 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 1006521 T931 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1392641283451/jetty7 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1392641283451/jetty7;done=false>>]
[junit4] 2> 1006521 T931 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1392641283451/jetty7
[junit4] 2> 1006521 T931 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1392641283451/jetty7/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1392641283451/jetty7/index;done=false>>]
[junit4] 2> 1006522 T931 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1392641283451/jetty7/index
[junit4] 2> 1006522 T931 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1006522 T931 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.method=testDistribSearch -Dtests.seed=91F782F333BE8A65 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=is -Dtests.timezone=Asia/Riyadh -Dtests.file.encoding=UTF-8
[junit4] FAILURE 58.1s J1 | ChaosMonkeyNothingIsSafeTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: document count mismatch. control=163 sum(shards)=164 cloudClient=164
[junit4] > at __randomizedtesting.SeedInfo.seed([91F782F333BE8A65:10110CEB44E1EA59]:0)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1230)
[junit4] > at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:208)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 1006526 T772 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Lucene46: {_version_=PostingsFormat(name=MockRandom), a_t=PostingsFormat(name=MockRandom), a_si=MockVariableIntBlock(baseBlockSize=29), id=PostingsFormat(name=Lucene41WithOrds)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=is, timezone=Asia/Riyadh
[junit4] 2> NOTE: Linux 3.8.0-35-generic amd64/Oracle Corporation 1.8.0 (64-bit)/cpus=8,threads=1,free=100138160,total=189792256
[junit4] 2> NOTE: All tests run in this JVM: [HighlighterTest, BlockCacheTest, DirectSolrConnectionTest, SolrCmdDistributorTest, SimplePostToolTest, TestQuerySenderListener, TestSolrXmlPersistence, TestCSVResponseWriter, TestBinaryResponseWriter, RAMDirectoryFactoryTest, IndexSchemaRuntimeFieldTest, ShardSplitTest, TestSolrXml, TestDocBasedVersionConstraints, TestDistributedGrouping, IndexBasedSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestFiltering, PreAnalyzedUpdateProcessorTest, TestFileDictionaryLookup, TestWordDelimiterFilterFactory, TestFoldingMultitermQuery, TestSolrCoreProperties, TestNRTOpen, LeaderElectionIntegrationTest, TestSweetSpotSimilarityFactory, TestDefaultSimilarityFactory, CachingDirectoryFactoryTest, TestSchemaNameResource, FileUtilsTest, ChaosMonkeyNothingIsSafeTest]
[junit4] Completed on J1 in 58.17s, 1 test, 1 failure <<< FAILURES!
[...truncated 735 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 362 suites, 1563 tests, 2 failures, 35 ignored (7 assumptions)
Total time: 57 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-fcs-b129 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure