You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/02/25 22:44:11 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60-ea-b04) -
Build # 9595 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9595/
Java: 32bit/jdk1.7.0_60-ea-b04 -server -XX:+UseSerialGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([7251B54D221CEEDC]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([7251B54D221CEEDC]:0)
Build Log:
[...truncated 11154 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> 529169 T2151 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 529169 T2151 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /khv/wa
[junit4] 2> 529173 T2151 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeySafeLeaderTest-1393357415240
[junit4] 2> 529173 T2151 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 529174 T2152 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 529274 T2151 oasc.ZkTestServer.run start zk server on port:58172
[junit4] 2> 529275 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529342 T2158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1229554 name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529343 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529343 T2151 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 529351 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529353 T2160 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c20f96 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529353 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529354 T2151 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 529358 T2151 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 529362 T2151 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 529366 T2151 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 529370 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 529371 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 529378 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 529379 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 529385 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 529385 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 529390 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 529391 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 529396 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 529397 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 529402 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 529403 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 529408 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 529408 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 529413 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 529414 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 529419 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 529419 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 529423 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 529424 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 529428 T2151 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 529429 T2151 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 529435 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529437 T2162 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2955eb name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529437 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529646 T2151 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 529649 T2151 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 529665 T2151 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:45401
[junit4] 2> 529666 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 529667 T2151 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 529667 T2151 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508
[junit4] 2> 529667 T2151 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/'
[junit4] 2> 529676 T2151 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/solr.xml
[junit4] 2> 529697 T2151 oasc.CoreContainer.<init> New CoreContainer 28838409
[junit4] 2> 529697 T2151 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/]
[junit4] 2> 529698 T2151 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 529698 T2151 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 529698 T2151 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 529699 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 529699 T2151 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 529699 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 529700 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 529700 T2151 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 529700 T2151 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 529703 T2151 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 529704 T2151 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 529704 T2151 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 529704 T2151 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58172/solr
[junit4] 2> 529705 T2151 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 529705 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529707 T2173 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fd7f6 name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529707 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529710 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 529711 T2175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1243546 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 529711 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 529713 T2151 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 529715 T2151 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 529719 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 529720 T2151 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45401_khv%2Fwa
[junit4] 2> 529722 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45401_khv%2Fwa
[junit4] 2> 529726 T2151 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 529727 T2151 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 529730 T2151 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:45401_khv%2Fwa
[junit4] 2> 529731 T2151 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 529732 T2151 oasc.Overseer.start Overseer (id=91315071565365252-127.0.0.1:45401_khv%2Fwa-n_0000000000) starting
[junit4] 2> 529735 T2151 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 529738 T2177 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 529738 T2151 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 529739 T2177 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 529743 T2151 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 529745 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 529747 T2176 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 529749 T2178 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 529750 T2178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 529750 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 529750 T2178 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 529752 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 529752 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45401/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:45401_khv%2Fwa",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 529752 T2176 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 529753 T2176 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 529754 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 529755 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 530751 T2178 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 530751 T2178 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1
[junit4] 2> 530752 T2178 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 530753 T2178 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 530753 T2178 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 530755 T2178 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 530756 T2178 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/'
[junit4] 2> 530756 T2178 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/lib/.svn/' to classloader
[junit4] 2> 530757 T2178 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/lib/classes/' to classloader
[junit4] 2> 530757 T2178 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/lib/README' to classloader
[junit4] 2> 530803 T2178 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 530824 T2178 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 530826 T2178 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 530831 T2178 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 531012 T2178 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 531013 T2178 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 531014 T2178 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 531019 T2178 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 531059 T2178 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 531060 T2178 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393357415508/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/
[junit4] 2> 531060 T2178 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f250ac
[junit4] 2> 531061 T2178 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data
[junit4] 2> 531062 T2178 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index/
[junit4] 2> 531062 T2178 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 531063 T2178 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index
[junit4] 2> 531064 T2178 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=97.7001953125, floorSegmentMB=0.65234375, forceMergeDeletesPctAllowed=25.842840998478867, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 531068 T2178 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 531068 T2178 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 531072 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 531072 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 531073 T2178 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 531073 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 531074 T2178 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 531074 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 531075 T2178 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 531075 T2178 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 531075 T2178 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 531076 T2178 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 531077 T2178 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 531077 T2178 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 531077 T2178 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 531078 T2178 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 531078 T2178 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 531079 T2178 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 531080 T2178 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 531083 T2178 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 531086 T2178 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 531087 T2178 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 531088 T2178 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 531089 T2178 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 531090 T2178 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 531090 T2178 oass.SolrIndexSearcher.<init> Opening Searcher@2ff84c[collection1] main
[junit4] 2> 531092 T2179 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2ff84c[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 531094 T2178 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 531094 T2151 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 531095 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 531094 T2182 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:45401/khv/wa collection:control_collection shard:shard1
[junit4] 2> 531097 T2182 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 531100 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 531101 T2184 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a32201 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 531102 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 531104 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 531108 T2182 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 531109 T2151 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 531111 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531111 T2182 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 531112 T2182 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2190 name=collection1 org.apache.solr.core.SolrCore@14a1ca0 url=https://127.0.0.1:45401/khv/wa/collection1 node=127.0.0.1:45401_khv%2Fwa C2190_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:45401/khv/wa, core=collection1, node_name=127.0.0.1:45401_khv%2Fwa}
[junit4] 2> 531112 T2182 C2190 P45401 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:45401/khv/wa/collection1/
[junit4] 2> 531113 T2182 C2190 P45401 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 531113 T2182 C2190 P45401 oasc.SyncStrategy.syncToMe https://127.0.0.1:45401/khv/wa/collection1/ has no replicas
[junit4] 2> 531113 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 531113 T2182 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:45401/khv/wa/collection1/ shard1
[junit4] 2> 531114 T2182 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 531116 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531118 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531124 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531228 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 531228 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 531270 T2182 oasc.ZkController.register We are https://127.0.0.1:45401/khv/wa/collection1/ and leader is https://127.0.0.1:45401/khv/wa/collection1/
[junit4] 2> 531270 T2182 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:45401/khv/wa
[junit4] 2> 531270 T2182 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 531270 T2182 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 531270 T2182 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 531271 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531271 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531271 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531272 T2182 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 531273 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 531273 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:45401/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:45401_khv%2Fwa",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 531275 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 531314 T2151 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 531315 T2151 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 531318 T2151 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 531319 T2151 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:34509
[junit4] 2> 531322 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 531322 T2151 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 531323 T2151 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177
[junit4] 2> 531324 T2151 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/'
[junit4] 2> 531342 T2151 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/solr.xml
[junit4] 2> 531364 T2151 oasc.CoreContainer.<init> New CoreContainer 1726962
[junit4] 2> 531365 T2151 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/]
[junit4] 2> 531365 T2151 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 531365 T2151 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 531366 T2151 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 531366 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 531366 T2151 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 531367 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 531367 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 531367 T2151 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 531367 T2151 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 531371 T2151 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 531371 T2151 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 531371 T2151 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 531372 T2151 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58172/solr
[junit4] 2> 531372 T2151 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 531373 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 531374 T2195 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b12097 name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 531375 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 531378 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 531378 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 531379 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 531380 T2197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e0dcf6 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 531380 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 531388 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 532393 T2151 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34509_khv%2Fwa
[junit4] 2> 532394 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34509_khv%2Fwa
[junit4] 2> 532397 T2184 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 532397 T2175 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 532397 T2197 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 532402 T2198 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 532403 T2198 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 532404 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 532404 T2198 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 532404 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 532404 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 532405 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 532406 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34509/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34509_khv%2Fwa",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 532406 T2176 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 532406 T2176 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 532408 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 532513 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 532513 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 532513 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 533404 T2198 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 533404 T2198 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1
[junit4] 2> 533405 T2198 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 533406 T2198 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 533406 T2198 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 533408 T2198 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 533408 T2198 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/'
[junit4] 2> 533409 T2198 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/lib/.svn/' to classloader
[junit4] 2> 533410 T2198 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/lib/classes/' to classloader
[junit4] 2> 533410 T2198 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/lib/README' to classloader
[junit4] 2> 533477 T2198 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 533501 T2198 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 533503 T2198 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 533508 T2198 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 533699 T2198 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 533700 T2198 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 533701 T2198 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 533705 T2198 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 533724 T2198 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 533724 T2198 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393357417177/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/
[junit4] 2> 533724 T2198 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f250ac
[junit4] 2> 533725 T2198 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1
[junit4] 2> 533725 T2198 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index/
[junit4] 2> 533726 T2198 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 533727 T2198 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index
[junit4] 2> 533728 T2198 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=97.7001953125, floorSegmentMB=0.65234375, forceMergeDeletesPctAllowed=25.842840998478867, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 533730 T2198 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 533730 T2198 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 533733 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 533733 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 533733 T2198 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 533734 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 533734 T2198 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 533734 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 533734 T2198 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 533735 T2198 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 533735 T2198 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 533735 T2198 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 533735 T2198 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 533736 T2198 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 533736 T2198 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 533736 T2198 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 533736 T2198 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 533737 T2198 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 533737 T2198 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 533740 T2198 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 533743 T2198 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 533743 T2198 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 533744 T2198 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 533745 T2198 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 533746 T2198 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 533746 T2198 oass.SolrIndexSearcher.<init> Opening Searcher@18216e3[collection1] main
[junit4] 2> 533750 T2199 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18216e3[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 533753 T2198 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 533754 T2151 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 533754 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 533754 T2202 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:34509/khv/wa collection:collection1 shard:shard3
[junit4] 2> 533756 T2202 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 533764 T2202 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
[junit4] 2> 533767 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533767 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533767 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533767 T2202 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 533768 T2202 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2191 name=collection1 org.apache.solr.core.SolrCore@8ac754 url=https://127.0.0.1:34509/khv/wa/collection1 node=127.0.0.1:34509_khv%2Fwa C2191_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:34509/khv/wa, core=collection1, node_name=127.0.0.1:34509_khv%2Fwa}
[junit4] 2> 533768 T2202 C2191 P34509 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:34509/khv/wa/collection1/
[junit4] 2> 533768 T2202 C2191 P34509 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 533768 T2202 C2191 P34509 oasc.SyncStrategy.syncToMe https://127.0.0.1:34509/khv/wa/collection1/ has no replicas
[junit4] 2> 533768 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 533768 T2202 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:34509/khv/wa/collection1/ shard3
[junit4] 2> 533769 T2202 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 533770 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533772 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533774 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533879 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 533879 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 533879 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 533923 T2202 oasc.ZkController.register We are https://127.0.0.1:34509/khv/wa/collection1/ and leader is https://127.0.0.1:34509/khv/wa/collection1/
[junit4] 2> 533923 T2202 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:34509/khv/wa
[junit4] 2> 533923 T2202 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 533923 T2202 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 533924 T2202 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 533924 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533925 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533925 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533925 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 533926 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 533926 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:34509/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34509_khv%2Fwa",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 533928 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 533954 T2151 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 533955 T2151 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 533958 T2151 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 533960 T2151 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:35642
[junit4] 2> 533963 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 533964 T2151 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 533964 T2151 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823
[junit4] 2> 533965 T2151 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/'
[junit4] 2> 533974 T2151 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/solr.xml
[junit4] 2> 533994 T2151 oasc.CoreContainer.<init> New CoreContainer 22431363
[junit4] 2> 533995 T2151 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/]
[junit4] 2> 533995 T2151 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 533996 T2151 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 533996 T2151 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 533996 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 533996 T2151 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 533997 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 533997 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 533997 T2151 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 533997 T2151 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 534000 T2151 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 534001 T2151 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 534001 T2151 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 534001 T2151 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58172/solr
[junit4] 2> 534002 T2151 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 534002 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 534004 T2213 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@126178f name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 534005 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 534007 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 534008 T2215 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@189ea73 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 534008 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 534014 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 534031 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 534031 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 534031 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 534031 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 535016 T2151 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35642_khv%2Fwa
[junit4] 2> 535018 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35642_khv%2Fwa
[junit4] 2> 535020 T2175 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 535020 T2184 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 535020 T2197 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 535020 T2215 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 535027 T2216 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 535027 T2216 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 535028 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 535028 T2216 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 535028 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 535029 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 535030 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 535030 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:35642/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35642_khv%2Fwa",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 535030 T2176 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 535031 T2176 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 535034 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 535137 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 535137 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 535137 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 535137 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536029 T2216 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 536029 T2216 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1
[junit4] 2> 536029 T2216 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 536031 T2216 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 536031 T2216 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 536033 T2216 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 536033 T2216 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/'
[junit4] 2> 536034 T2216 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/lib/.svn/' to classloader
[junit4] 2> 536035 T2216 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/lib/classes/' to classloader
[junit4] 2> 536035 T2216 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/lib/README' to classloader
[junit4] 2> 536081 T2216 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 536104 T2216 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 536106 T2216 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 536111 T2216 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 536326 T2216 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 536328 T2216 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 536329 T2216 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 536333 T2216 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 536355 T2216 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 536356 T2216 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393357419823/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/
[junit4] 2> 536356 T2216 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f250ac
[junit4] 2> 536357 T2216 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2
[junit4] 2> 536357 T2216 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index/
[junit4] 2> 536358 T2216 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 536359 T2216 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index
[junit4] 2> 536359 T2216 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=97.7001953125, floorSegmentMB=0.65234375, forceMergeDeletesPctAllowed=25.842840998478867, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 536363 T2216 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 536363 T2216 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 536368 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 536369 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 536369 T2216 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 536369 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 536370 T2216 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 536370 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 536371 T2216 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 536371 T2216 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 536372 T2216 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 536373 T2216 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 536373 T2216 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 536374 T2216 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 536374 T2216 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 536375 T2216 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 536375 T2216 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 536376 T2216 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 536377 T2216 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 536381 T2216 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 536384 T2216 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 536384 T2216 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 536385 T2216 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 536387 T2216 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 536387 T2216 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 536387 T2216 oass.SolrIndexSearcher.<init> Opening Searcher@4488d8[collection1] main
[junit4] 2> 536389 T2217 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4488d8[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 536390 T2216 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 536391 T2151 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 536392 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 536392 T2220 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:35642/khv/wa collection:collection1 shard:shard2
[junit4] 2> 536394 T2220 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 536401 T2220 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 536404 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536405 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536405 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536405 T2220 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 536406 T2220 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2192 name=collection1 org.apache.solr.core.SolrCore@894fe3 url=https://127.0.0.1:35642/khv/wa/collection1 node=127.0.0.1:35642_khv%2Fwa C2192_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:35642/khv/wa, core=collection1, node_name=127.0.0.1:35642_khv%2Fwa}
[junit4] 2> 536406 T2220 C2192 P35642 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:35642/khv/wa/collection1/
[junit4] 2> 536406 T2220 C2192 P35642 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 536406 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 536407 T2220 C2192 P35642 oasc.SyncStrategy.syncToMe https://127.0.0.1:35642/khv/wa/collection1/ has no replicas
[junit4] 2> 536407 T2220 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:35642/khv/wa/collection1/ shard2
[junit4] 2> 536407 T2220 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 536409 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536412 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536416 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536519 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536519 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536519 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536519 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536563 T2220 oasc.ZkController.register We are https://127.0.0.1:35642/khv/wa/collection1/ and leader is https://127.0.0.1:35642/khv/wa/collection1/
[junit4] 2> 536563 T2220 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:35642/khv/wa
[junit4] 2> 536563 T2220 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 536563 T2220 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 536563 T2220 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 536564 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536564 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536565 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536565 T2220 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 536566 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 536566 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:35642/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35642_khv%2Fwa",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 536568 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 536598 T2151 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 536599 T2151 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 536603 T2151 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 536605 T2151 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55287
[junit4] 2> 536608 T2151 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 536609 T2151 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 536609 T2151 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461
[junit4] 2> 536610 T2151 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/'
[junit4] 2> 536628 T2151 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/solr.xml
[junit4] 2> 536648 T2151 oasc.CoreContainer.<init> New CoreContainer 15609678
[junit4] 2> 536648 T2151 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/]
[junit4] 2> 536649 T2151 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 536649 T2151 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 536649 T2151 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 536650 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 536650 T2151 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 536650 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 536650 T2151 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 536651 T2151 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 536651 T2151 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 536654 T2151 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 536655 T2151 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 536655 T2151 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 536655 T2151 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58172/solr
[junit4] 2> 536656 T2151 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 536656 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 536659 T2231 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1533afd name:ZooKeeperConnection Watcher:127.0.0.1:58172 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 536659 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 536663 T2151 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 536664 T2233 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1edbb54 name:ZooKeeperConnection Watcher:127.0.0.1:58172/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 536664 T2151 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 536668 T2151 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 536670 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536670 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536670 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536670 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 536670 T2233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 537676 T2151 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55287_khv%2Fwa
[junit4] 2> 537679 T2151 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55287_khv%2Fwa
[junit4] 2> 537683 T2184 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537683 T2215 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537683 T2197 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537684 T2175 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537683 T2233 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 537695 T2234 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 537696 T2234 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 537698 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 537698 T2234 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 537698 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 537699 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 537700 T2176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 537701 T2176 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:55287/khv/wa",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:55287_khv%2Fwa",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 537701 T2176 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 537702 T2176 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 537706 T2175 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 537809 T2233 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 537809 T2197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 537809 T2184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 537809 T2215 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 537809 T2175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 538698 T2234 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 538699 T2234 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1
[junit4] 2> 538699 T2234 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 538700 T2234 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 538701 T2234 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 538703 T2234 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 538703 T2234 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/'
[junit4] 2> 538704 T2234 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/lib/.svn/' to classloader
[junit4] 2> 538705 T2234 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/lib/classes/' to classloader
[junit4] 2> 538705 T2234 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/lib/README' to classloader
[junit4] 2> 538748 T2234 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 538768 T2234 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 538770 T2234 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 538775 T2234 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 538968 T2234 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 538970 T2234 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 538971 T2234 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 538974 T2234 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 538989 T2234 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 538990 T2234 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393357422461/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/
[junit4] 2> 538990 T2234 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f250ac
[junit4] 2> 538990 T2234 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3
[junit4] 2> 538991 T2234 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/index/
[junit4] 2> 538991 T2234 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 538992 T2234 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/index
[junit4] 2> 538992 T2234 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=97.7001953125, floorSegmentMB=0.65234375, forceMergeDeletesPctAllowed=25.842840998478867, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 538995 T2234 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393357415240/jetty3/index,segFN=segments_1,generation=1}
[junit4] 2> 538995 T2234 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 538999 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 538999 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 538999 T2234 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 539000 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 539000 T2234 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 539000 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 539001 T2234 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 539001 T2234 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 539001 T2234 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 539002 T2234 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 539002 T2234 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 539003 T2234 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 539003 T2234 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 53
[...truncated too long message...]
2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 67) Thread[id=3061, name=Thread-1572, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4] 2> at org.apache.solr.core.CloserThread.run(CoreContainer.java:1018)
[junit4] 2> 68) Thread[id=3235, name=qtp15937844-3235 Selector0, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:193)
[junit4] 2> at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
[junit4] 2> at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:78)
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4] 2> at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:536)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 69) Thread[id=3050, name=qtp9333592-3050 Acceptor0 SslSelectChannelConnector@127.0.0.1:35928, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 70) Thread[id=3051, name=qtp9333592-3051 Acceptor1 SslSelectChannelConnector@127.0.0.1:35928, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 71) Thread[id=2595, name=qtp20914976-2595 Selector1, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.nio.ch.IOUtil.drain(Native Method)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4] 2> at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:536)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 72) Thread[id=2597, name=qtp20914976-2597 Acceptor1 SslSelectChannelConnector@127.0.0.1:52388, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 73) Thread[id=2934, name=Thread-1492, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4] 2> at org.apache.solr.core.CloserThread.run(CoreContainer.java:1018)
[junit4] 2> 74) Thread[id=3066, name=HashSessionScavenger-102, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4] 2> NOTE: test params are: codec=Lucene46: {a_si=PostingsFormat(name=NestedPulsing), _version_=PostingsFormat(name=NestedPulsing), id=PostingsFormat(name=Direct), a_t=PostingsFormat(name=Lucene41VarGapFixedInterval), rnd_b=PostingsFormat(name=Direct)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=et, timezone=Etc/GMT+4
[junit4] 2> NOTE: Linux 3.8.0-36-generic i386/Oracle Corporation 1.7.0_60-ea (32-bit)/cpus=8,threads=85,free=324068616,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [IndexSchemaTest, TestUniqueKeyFieldResource, TestDefaultSearchFieldResource, BasicFunctionalityTest, CSVRequestHandlerTest, ClusterStateUpdateTest, DistributedSpellCheckComponentTest, HdfsSyncSliceTest, ExternalFileFieldSortTest, UpdateParamsTest, TestJmxMonitoredMap, TestConfig, CoreAdminHandlerTest, AnalysisAfterCoreReloadTest, TestSystemIdResolver, ScriptEngineTest, EchoParamsTest, TestTrie, TermsComponentTest, TestSolrDeletionPolicy1, TestElisionMultitermQuery, DocValuesMissingTest, TestPostingsSolrHighlighter, LukeRequestHandlerTest, TestSchemaVersionResource, TestRandomDVFaceting, AnalysisErrorHandlingTest, RegexBoostProcessorTest, TestRemoteStreaming, TestStressUserVersions, FastVectorHighlighterTest, TriLevelCompositeIdRoutingTest, SuggesterTest, SpellPossibilityIteratorTest, FunctionTest, CursorPagingTest, DistributedSuggestComponentTest, QueryElevationComponentTest, SpatialFilterTest, TestSolrQueryParserResource, PeerSyncTest, TestInitQParser, SliceStateUpdateTest, DateMathParserTest, TestAddFieldRealTimeGet, SoftAutoCommitTest, TestMergePolicyConfig, BadCopyFieldTest, TestSuggestSpellingConverter, SolrInfoMBeanTest, ResponseLogComponentTest, SearchHandlerTest, TestInfoStreamLogging, InfoHandlerTest, CoreAdminCreateDiscoverTest, BinaryUpdateRequestHandlerTest, TermVectorComponentTest, TestPseudoReturnFields, QueryResultKeyTest, SampleTest, TestGroupingSearch, TestPerFieldSimilarity, TestRealTimeGet, AbstractAnalyticsStatsTest, DirectUpdateHandlerOptimizeTest, DeleteShardTest, TestLMDirichletSimilarityFactory, TestCopyFieldCollectionResource, HdfsDirectoryTest, TestCoreContainer, ReturnFieldsTest, LegacyHTMLStripCharFilterTest, BasicDistributedZkTest, TestLMJelinekMercerSimilarityFactory, DOMUtilTest, TestQueryUtils, TestSolrIndexConfig, ExpressionTest, ShowFileRequestHandlerTest, TestSurroundQueryParser, DistributedDebugComponentTest, FieldMutatingUpdateProcessorTest, BlockCacheTest, TestLuceneMatchVersion, TestExtendedDismaxParser, TestCollapseQParserPlugin, TestNonNRTOpen, MBeansHandlerTest, ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=7251B54D221CEEDC -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=et -Dtests.timezone=Etc/GMT+4 -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([7251B54D221CEEDC]:0)
[junit4] Completed on J0 in 7224.51s, 1 test, 2 errors <<< FAILURES!
[...truncated 11 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:465: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:445: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1274: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 373 suites, 1606 tests, 1 suite-level error, 1 error, 35 ignored (7 assumptions)
Total time: 149 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b04 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0-fcs-b129) - Build
# 9597 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9597/
Java: 64bit/jdk1.8.0-fcs-b129 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch
Error Message:
commitWithin did not work expected:<68> but was:<67>
Stack Trace:
java.lang.AssertionError: commitWithin did not work expected:<68> but was:<67>
at __randomizedtesting.SeedInfo.seed([4CDAA86DE1E3E202:CD3C267596BC823E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:354)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Build Log:
[...truncated 10430 lines...]
[junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4] 2> 643277 T2745 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 643277 T2745 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /pf_uir/o
[junit4] 2> 643280 T2745 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1393378926844
[junit4] 2> 643280 T2745 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 643281 T2746 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 643381 T2745 oasc.ZkTestServer.run start zk server on port:37619
[junit4] 2> 643381 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 643417 T2752 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@466897a7 name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 643417 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 643417 T2745 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 643421 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 643422 T2754 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5546fdef name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 643422 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 643422 T2745 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 643424 T2745 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 643427 T2745 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 643429 T2745 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 643432 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 643432 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 643435 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 643435 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 643438 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 643438 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 643440 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 643440 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 643442 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 643442 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 643444 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 643444 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 643446 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 643446 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 643448 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 643449 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 643450 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 643451 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 643452 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 643453 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 643455 T2745 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 643455 T2745 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 643659 T2745 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 643661 T2745 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44752
[junit4] 2> 643662 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 643662 T2745 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 643662 T2745 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022
[junit4] 2> 643663 T2745 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/'
[junit4] 2> 643671 T2745 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/solr.xml
[junit4] 2> 643677 T2745 oasc.CoreContainer.<init> New CoreContainer 1495401464
[junit4] 2> 643677 T2745 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/]
[junit4] 2> 643677 T2745 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 643678 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 643679 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 643679 T2745 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 643679 T2745 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 643685 T2745 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 643685 T2745 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 643686 T2745 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 643686 T2745 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37619/solr
[junit4] 2> 643686 T2745 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 643687 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 643688 T2765 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61a9a631 name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 643688 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 643690 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 643691 T2767 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d934628 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 643691 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 643692 T2745 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 643694 T2745 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 643696 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 643697 T2745 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44752_pf_uir%2Fo
[junit4] 2> 643697 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44752_pf_uir%2Fo
[junit4] 2> 643699 T2745 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 643700 T2745 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 643702 T2745 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:44752_pf_uir%2Fo
[junit4] 2> 643702 T2745 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 643706 T2745 oasc.Overseer.start Overseer (id=91316481349779459-127.0.0.1:44752_pf_uir%2Fo-n_0000000000) starting
[junit4] 2> 643709 T2745 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 643711 T2769 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 643712 T2745 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 643712 T2769 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 643713 T2745 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 643714 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 643715 T2768 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 643716 T2770 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 643716 T2770 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 643718 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 643718 T2770 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 643719 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 643720 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44752/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:44752_pf_uir%2Fo",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 643720 T2768 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 643720 T2768 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 643721 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 643722 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 644718 T2770 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 644718 T2770 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1
[junit4] 2> 644718 T2770 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 644719 T2770 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 644719 T2770 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 644720 T2770 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 644720 T2770 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/'
[junit4] 2> 644721 T2770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/lib/.svn/' to classloader
[junit4] 2> 644721 T2770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/lib/classes/' to classloader
[junit4] 2> 644721 T2770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/lib/README' to classloader
[junit4] 2> 644735 T2770 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 644742 T2770 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 644743 T2770 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 644746 T2770 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 644828 T2770 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 644831 T2770 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 644832 T2770 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 644840 T2770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 644842 T2770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 644844 T2770 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 644846 T2770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 644846 T2770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 644846 T2770 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 644847 T2770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 644847 T2770 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 644847 T2770 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 644847 T2770 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393378927022/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data/
[junit4] 2> 644848 T2770 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4a7385
[junit4] 2> 644848 T2770 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data
[junit4] 2> 644848 T2770 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data/index/
[junit4] 2> 644849 T2770 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 644849 T2770 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/control/data/index
[junit4] 2> 644849 T2770 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=2.447265625, floorSegmentMB=1.44921875, forceMergeDeletesPctAllowed=3.3196981008770865, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1240017361539163
[junit4] 2> 644849 T2770 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@53a74e8f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@736a53af),segFN=segments_1,generation=1}
[junit4] 2> 644850 T2770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 644851 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 644851 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 644851 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 644852 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 644852 T2770 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 644853 T2770 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 644854 T2770 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 644854 T2770 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 644854 T2770 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 644855 T2770 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 644856 T2770 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 644857 T2770 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 644857 T2770 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2564475347007128]
[junit4] 2> 644857 T2770 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@53a74e8f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@736a53af),segFN=segments_1,generation=1}
[junit4] 2> 644857 T2770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 644857 T2770 oass.SolrIndexSearcher.<init> Opening Searcher@4de84a8d[collection1] main
[junit4] 2> 644859 T2771 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4de84a8d[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 644860 T2770 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 644860 T2745 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 644860 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 644860 T2774 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44752/pf_uir/o collection:control_collection shard:shard1
[junit4] 2> 644861 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 644862 T2776 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32ffa0ca name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 644862 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 644863 T2774 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 644863 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 644865 T2745 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 644867 T2774 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 644868 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 644869 T2774 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 644869 T2774 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1053 name=collection1 org.apache.solr.core.SolrCore@4bab5be0 url=http://127.0.0.1:44752/pf_uir/o/collection1 node=127.0.0.1:44752_pf_uir%2Fo C1053_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:44752/pf_uir/o, core=collection1, node_name=127.0.0.1:44752_pf_uir%2Fo}
[junit4] 2> 644869 T2774 C1053 P44752 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44752/pf_uir/o/collection1/
[junit4] 2> 644869 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 644869 T2774 C1053 P44752 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 644869 T2774 C1053 P44752 oasc.SyncStrategy.syncToMe http://127.0.0.1:44752/pf_uir/o/collection1/ has no replicas
[junit4] 2> 644870 T2774 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44752/pf_uir/o/collection1/ shard1
[junit4] 2> 644870 T2774 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 644871 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 644873 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 644878 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 644980 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 644980 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 645024 T2774 oasc.ZkController.register We are http://127.0.0.1:44752/pf_uir/o/collection1/ and leader is http://127.0.0.1:44752/pf_uir/o/collection1/
[junit4] 2> 645024 T2774 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44752/pf_uir/o
[junit4] 2> 645025 T2774 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 645025 T2774 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 645025 T2774 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 645026 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 645026 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 645026 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 645026 T2774 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 645027 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 645028 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:44752/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:44752_pf_uir%2Fo",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 645031 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 645069 T2745 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 645069 T2745 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 645071 T2745 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43774
[junit4] 2> 645072 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 645072 T2745 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 645072 T2745 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429
[junit4] 2> 645072 T2745 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/'
[junit4] 2> 645081 T2745 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/solr.xml
[junit4] 2> 645092 T2745 oasc.CoreContainer.<init> New CoreContainer 1388692918
[junit4] 2> 645092 T2745 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/]
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 645093 T2745 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 645094 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 645094 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 645094 T2745 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 645094 T2745 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 645096 T2745 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 645096 T2745 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 645096 T2745 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 645096 T2745 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37619/solr
[junit4] 2> 645097 T2745 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 645097 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 645098 T2787 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40a0ca8c name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 645099 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 645100 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 645101 T2789 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74458bb8 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 645101 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 645103 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 645132 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 645132 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 645132 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 646106 T2745 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43774_pf_uir%2Fo
[junit4] 2> 646107 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43774_pf_uir%2Fo
[junit4] 2> 646108 T2776 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 646108 T2789 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 646108 T2767 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 646112 T2790 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 646112 T2790 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 646112 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 646112 T2790 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 646112 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 646113 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 646113 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 646113 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43774/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:43774_pf_uir%2Fo",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 646114 T2768 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 646114 T2768 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 646115 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 646220 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 646220 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 646220 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647113 T2790 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 647113 T2790 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1
[junit4] 2> 647113 T2790 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 647114 T2790 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 647114 T2790 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 647115 T2790 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 647115 T2790 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/'
[junit4] 2> 647115 T2790 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/lib/.svn/' to classloader
[junit4] 2> 647116 T2790 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/lib/classes/' to classloader
[junit4] 2> 647116 T2790 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/lib/README' to classloader
[junit4] 2> 647130 T2790 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 647137 T2790 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 647138 T2790 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 647142 T2790 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 647238 T2790 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 647241 T2790 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 647242 T2790 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 647250 T2790 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 647252 T2790 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 647253 T2790 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 647254 T2790 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 647254 T2790 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 647254 T2790 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 647255 T2790 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 647255 T2790 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 647255 T2790 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 647255 T2790 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393378928429/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1/
[junit4] 2> 647255 T2790 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4a7385
[junit4] 2> 647256 T2790 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1
[junit4] 2> 647256 T2790 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1/index/
[junit4] 2> 647256 T2790 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 647256 T2790 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty1/index
[junit4] 2> 647257 T2790 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=2.447265625, floorSegmentMB=1.44921875, forceMergeDeletesPctAllowed=3.3196981008770865, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1240017361539163
[junit4] 2> 647257 T2790 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@46c55a5e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44bcea69),segFN=segments_1,generation=1}
[junit4] 2> 647257 T2790 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 647259 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 647259 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 647259 T2790 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 647260 T2790 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 647260 T2790 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 647261 T2790 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 647261 T2790 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 647261 T2790 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 647261 T2790 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 647262 T2790 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 647262 T2790 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 647262 T2790 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 647262 T2790 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 647264 T2790 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 647265 T2790 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 647265 T2790 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 647265 T2790 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2564475347007128]
[junit4] 2> 647265 T2790 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@46c55a5e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44bcea69),segFN=segments_1,generation=1}
[junit4] 2> 647266 T2790 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 647266 T2790 oass.SolrIndexSearcher.<init> Opening Searcher@7bc891fa[collection1] main
[junit4] 2> 647267 T2791 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7bc891fa[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 647268 T2790 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 647269 T2745 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 647269 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 647269 T2794 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43774/pf_uir/o collection:collection1 shard:shard2
[junit4] 2> 647270 T2794 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 647274 T2794 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 647275 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647275 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647275 T2794 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 647275 T2794 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 647275 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> ASYNC NEW_CORE C1054 name=collection1 org.apache.solr.core.SolrCore@c915954 url=http://127.0.0.1:43774/pf_uir/o/collection1 node=127.0.0.1:43774_pf_uir%2Fo C1054_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:43774/pf_uir/o, core=collection1, node_name=127.0.0.1:43774_pf_uir%2Fo}
[junit4] 2> 647275 T2794 C1054 P43774 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43774/pf_uir/o/collection1/
[junit4] 2> 647276 T2794 C1054 P43774 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 647276 T2794 C1054 P43774 oasc.SyncStrategy.syncToMe http://127.0.0.1:43774/pf_uir/o/collection1/ has no replicas
[junit4] 2> 647276 T2794 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43774/pf_uir/o/collection1/ shard2
[junit4] 2> 647276 T2794 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 647276 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 647279 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647280 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647282 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647384 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647384 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647384 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647431 T2794 oasc.ZkController.register We are http://127.0.0.1:43774/pf_uir/o/collection1/ and leader is http://127.0.0.1:43774/pf_uir/o/collection1/
[junit4] 2> 647431 T2794 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43774/pf_uir/o
[junit4] 2> 647431 T2794 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 647431 T2794 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 647431 T2794 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 647432 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647432 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647433 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647433 T2794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 647433 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 647434 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:43774/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:43774_pf_uir%2Fo",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 647435 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 647471 T2745 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 647472 T2745 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 647473 T2745 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49913
[junit4] 2> 647474 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 647475 T2745 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 647475 T2745 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834
[junit4] 2> 647475 T2745 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/'
[junit4] 2> 647484 T2745 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/solr.xml
[junit4] 2> 647494 T2745 oasc.CoreContainer.<init> New CoreContainer 1016064984
[junit4] 2> 647494 T2745 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/]
[junit4] 2> 647495 T2745 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 647495 T2745 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 647495 T2745 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 647495 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 647496 T2745 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 647498 T2745 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 647498 T2745 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 647499 T2745 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 647499 T2745 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37619/solr
[junit4] 2> 647499 T2745 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 647500 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 647501 T2805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11091dd0 name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 647501 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 647502 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 647503 T2807 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a7585f9 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 647503 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 647506 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 647537 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647537 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647537 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 647537 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 648508 T2745 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49913_pf_uir%2Fo
[junit4] 2> 648509 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49913_pf_uir%2Fo
[junit4] 2> 648510 T2776 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 648511 T2767 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 648510 T2789 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 648511 T2807 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 648513 T2808 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 648514 T2808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 648514 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 648515 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 648514 T2808 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 648515 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 648516 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 648516 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49913/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49913_pf_uir%2Fo",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 648516 T2768 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 648516 T2768 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 648518 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 648696 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 648696 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 648696 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 648696 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649515 T2808 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 649515 T2808 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1
[junit4] 2> 649515 T2808 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 649516 T2808 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 649516 T2808 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 649517 T2808 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 649517 T2808 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/'
[junit4] 2> 649518 T2808 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/lib/.svn/' to classloader
[junit4] 2> 649518 T2808 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/lib/classes/' to classloader
[junit4] 2> 649518 T2808 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/lib/README' to classloader
[junit4] 2> 649532 T2808 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 649540 T2808 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 649541 T2808 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 649545 T2808 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 649632 T2808 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 649636 T2808 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 649637 T2808 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 649644 T2808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 649646 T2808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 649648 T2808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 649648 T2808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 649648 T2808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 649649 T2808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 649649 T2808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 649649 T2808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 649650 T2808 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 649650 T2808 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393378930834/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2/
[junit4] 2> 649650 T2808 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4a7385
[junit4] 2> 649650 T2808 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2
[junit4] 2> 649651 T2808 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2/index/
[junit4] 2> 649651 T2808 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 649651 T2808 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty2/index
[junit4] 2> 649651 T2808 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=2.447265625, floorSegmentMB=1.44921875, forceMergeDeletesPctAllowed=3.3196981008770865, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1240017361539163
[junit4] 2> 649652 T2808 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@22aa45db lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3cebfafb),segFN=segments_1,generation=1}
[junit4] 2> 649652 T2808 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 649654 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 649655 T2808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 649655 T2808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 649655 T2808 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 649655 T2808 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 649655 T2808 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 649656 T2808 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 649656 T2808 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 649656 T2808 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 649656 T2808 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 649657 T2808 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 649657 T2808 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 649658 T2808 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 649659 T2808 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 649659 T2808 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 649660 T2808 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2564475347007128]
[junit4] 2> 649660 T2808 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@22aa45db lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3cebfafb),segFN=segments_1,generation=1}
[junit4] 2> 649660 T2808 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 649660 T2808 oass.SolrIndexSearcher.<init> Opening Searcher@21096384[collection1] main
[junit4] 2> 649662 T2809 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@21096384[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 649663 T2808 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 649664 T2745 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 649664 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 649664 T2812 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49913/pf_uir/o collection:collection1 shard:shard1
[junit4] 2> 649665 T2812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 649669 T2812 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 649670 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649670 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649670 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649671 T2812 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 649671 T2812 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 649671 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C1055 name=collection1 org.apache.solr.core.SolrCore@ed5308d url=http://127.0.0.1:49913/pf_uir/o/collection1 node=127.0.0.1:49913_pf_uir%2Fo C1055_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:49913/pf_uir/o, core=collection1, node_name=127.0.0.1:49913_pf_uir%2Fo}
[junit4] 2> 649671 T2812 C1055 P49913 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49913/pf_uir/o/collection1/
[junit4] 2> 649672 T2812 C1055 P49913 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 649672 T2812 C1055 P49913 oasc.SyncStrategy.syncToMe http://127.0.0.1:49913/pf_uir/o/collection1/ has no replicas
[junit4] 2> 649672 T2812 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49913/pf_uir/o/collection1/ shard1
[junit4] 2> 649672 T2812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 649673 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649675 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649678 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649782 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649782 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649782 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649782 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649826 T2812 oasc.ZkController.register We are http://127.0.0.1:49913/pf_uir/o/collection1/ and leader is http://127.0.0.1:49913/pf_uir/o/collection1/
[junit4] 2> 649826 T2812 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49913/pf_uir/o
[junit4] 2> 649826 T2812 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 649827 T2812 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 649827 T2812 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 649827 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649828 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649828 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649828 T2812 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 649829 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 649829 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:49913/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49913_pf_uir%2Fo",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 649830 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 649867 T2745 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 649868 T2745 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 649870 T2745 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60527
[junit4] 2> 649870 T2745 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 649871 T2745 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 649871 T2745 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229
[junit4] 2> 649871 T2745 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/'
[junit4] 2> 649887 T2745 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/solr.xml
[junit4] 2> 649892 T2745 oasc.CoreContainer.<init> New CoreContainer 1546132380
[junit4] 2> 649893 T2745 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/]
[junit4] 2> 649893 T2745 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 649893 T2745 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 649893 T2745 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 649894 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 649894 T2745 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 649894 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 649894 T2745 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 649895 T2745 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 649895 T2745 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 649896 T2745 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 649896 T2745 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 649897 T2745 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 649897 T2745 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37619/solr
[junit4] 2> 649897 T2745 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 649898 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 649899 T2823 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@715c94cf name:ZooKeeperConnection Watcher:127.0.0.1:37619 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 649899 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 649901 T2745 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 649901 T2825 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33846361 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 649902 T2745 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 649905 T2745 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 649932 T2825 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649932 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649932 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649932 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 649932 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 650907 T2745 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60527_pf_uir%2Fo
[junit4] 2> 650908 T2745 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60527_pf_uir%2Fo
[junit4] 2> 650910 T2789 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650910 T2767 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650910 T2776 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650910 T2825 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650910 T2807 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 650914 T2826 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 650914 T2826 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 650967 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 650967 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 650967 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 650967 T2826 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 650968 T2768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 650969 T2768 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:60527/pf_uir/o",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:60527_pf_uir%2Fo",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 650969 T2768 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 650969 T2768 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 650972 T2767 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 651074 T2789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651074 T2776 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651074 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651074 T2825 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651074 T2767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651968 T2826 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 651968 T2826 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/collection1
[junit4] 2> 651968 T2826 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 651969 T2826 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 651969 T2826 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 651970 T2826 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 651970 T2826 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/collection1/'
[junit4] 2> 651970 T2826 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393378933229/collection1/lib/.svn/' to classloader
[junit4] 2> 651970 T2826 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jett
[...truncated too long message...]
111 T2844 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 667111 T2825 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 667111 T2844 C1081 P55406 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@10648e85 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@185599ef),segFN=segments_7,generation=7}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@10648e85 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@185599ef),segFN=segments_8,generation=8}
[junit4] 2> 667112 T2844 C1081 P55406 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 8
[junit4] 2> 667112 T2825 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 667113 T2844 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4/index;done=false>>]
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4/index
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4;done=false>>]
[junit4] 2> 667113 T2844 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZkTest-1393378926844/jetty4
[junit4] 2> 667114 T2844 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 667114 T2844 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 667202 T2866 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4109d128 name:ZooKeeperConnection Watcher:127.0.0.1:37619/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 667203 T2866 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=4CDAA86DE1E3E202 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de_CH -Dtests.timezone=Pacific/Auckland -Dtests.file.encoding=UTF-8
[junit4] FAILURE 23.9s J0 | BasicDistributedZkTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: commitWithin did not work expected:<68> but was:<67>
[junit4] > at __randomizedtesting.SeedInfo.seed([4CDAA86DE1E3E202:CD3C267596BC823E]:0)
[junit4] > at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:354)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] 2> 667211 T2745 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 23943 T2744 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=de_CH, timezone=Pacific/Auckland
[junit4] 2> NOTE: Linux 3.8.0-36-generic amd64/Oracle Corporation 1.8.0 (64-bit)/cpus=8,threads=1,free=116480720,total=244842496
[junit4] 2> NOTE: All tests run in this JVM: [TestLMJelinekMercerSimilarityFactory, TestRemoteStreaming, TestCoreContainer, SchemaVersionSpecificBehaviorTest, TestFieldTypeCollectionResource, TestHighFrequencyDictionaryFactory, HdfsCollectionsAPIDistributedZkTest, SuggesterFSTTest, TestMergePolicyConfig, UUIDFieldTest, TestWriterPerf, AssignTest, TestDocBasedVersionConstraints, RecoveryZkTest, ZkCLITest, DateMathParserTest, ExternalFileFieldSortTest, CacheHeaderTest, PeerSyncTest, IndexSchemaTest, SynonymTokenizerTest, SuggestComponentTest, TestSolrXMLSerializer, TestManagedSchema, TestSort, UpdateRequestProcessorFactoryTest, TestFreeTextSuggestions, SolrTestCaseJ4Test, TestInfoStreamLogging, ScriptEngineTest, TestSerializedLuceneMatchVersion, TestDefaultSearchFieldResource, SolrInfoMBeanTest, TestRecoveryHdfs, DOMUtilTest, SuggesterTest, CopyFieldTest, SliceStateUpdateTest, TestLFUCache, TestTrie, TestFieldResource, TestShardHandlerFactory, TestDocSet, SearchHandlerTest, OverseerRolesTest, AnalysisAfterCoreReloadTest, EchoParamsTest, TestRandomDVFaceting, SampleTest, TestSchemaSimilarityResource, TestConfig, QueryElevationComponentTest, TestCollationFieldDocValues, ShardSplitTest, SimpleFacetsTest, FileBasedSpellCheckerTest, TestBadConfig, ClusterStateTest, TestSolrDeletionPolicy1, ResourceLoaderTest, SpatialFilterTest, TriLevelCompositeIdRoutingTest, AbstractAnalyticsStatsTest, TestPseudoReturnFields, DistributedQueryComponentCustomSortTest, TestJmxMonitoredMap, TestCollapseQParserPlugin, CoreAdminCreateDiscoverTest, TestJoin, BasicFunctionalityTest, HdfsChaosMonkeySafeLeaderTest, TestFunctionQuery, DeleteShardTest, TestAnalyzeInfixSuggestions, UnloadDistributedZkTest, SpellPossibilityIteratorTest, StatelessScriptUpdateProcessorFactoryTest, LeaderElectionTest, TestImplicitCoreProperties, SOLR749Test, TestDocumentBuilder, CSVRequestHandlerTest, DistributedDebugComponentTest, DistributedTermsComponentTest, LegacyHTMLStripCharFilterTest, TestSchemaResource, TestPostingsSolrHighlighter, TestSolrQueryParserDefaultOperatorResource, TermVectorComponentTest, BasicDistributedZkTest]
[junit4] Completed on J0 in 24.97s, 1 test, 1 failure <<< FAILURES!
[...truncated 651 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:465: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:445: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1274: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 373 suites, 1606 tests, 1 failure, 35 ignored (7 assumptions)
Total time: 47 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-fcs-b129 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-fcs-b129) - Build
# 9596 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9596/
Java: 32bit/jdk1.8.0-fcs-b129 -server -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED: org.apache.solr.client.solrj.SolrExampleBinaryTest.testUnicode
Error Message:
IOException occured when talking to server at: https://127.0.0.1:39662/solr/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:39662/solr/collection1
at __randomizedtesting.SeedInfo.seed([CA32FB715F20BCA7:D3FE4F7BFD9A6D50]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:510)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54)
at org.apache.solr.client.solrj.SolrExampleTests.testUnicode(SolrExampleTests.java:370)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:395)
... 45 more
FAILED: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testShutdown
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
at __randomizedtesting.SeedInfo.seed([CA32FB715F20BCA7:2944F2E4B85A29D5]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:200)
at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:771)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:222)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
... 47 more
Build Log:
[...truncated 11458 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
[junit4] 2> 5100 T23 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 5100 T23 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 5103 T23 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
[junit4] 2> 5111 T23 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1393373232300
[junit4] 2> 5115 T23 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 5118 T24 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 5217 T23 oasc.ZkTestServer.run start zk server on port:45644
[junit4] 2> 5265 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 71927 T27 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 66639ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 71929 T23 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
[junit4] 2> 71950 T25 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446b843ceb0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 71953 T23 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:45644 45644
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CloudSolrServerTest -Dtests.method=testShutdown -Dtests.seed=CA32FB715F20BCA7 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=Asia/Sakhalin -Dtests.file.encoding=US-ASCII
[junit4] ERROR 67.1s J0 | CloudSolrServerTest.testShutdown <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
[junit4] > at __randomizedtesting.SeedInfo.seed([CA32FB715F20BCA7:2944F2E4B85A29D5]:0)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:148)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:94)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:85)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89)
[junit4] > at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.setUp(AbstractDistribZkTestBase.java:70)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.setUp(AbstractFullDistribZkTestBase.java:200)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServerTest.setUp(CloudSolrServerTest.java:77)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:45644 within 30000 ms
[junit4] > at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:222)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:140)
[junit4] > ... 47 more
[junit4] 2> 72239 T23 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1393373299430
[junit4] 2> 72242 T23 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 72243 T31 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 72343 T23 oasc.ZkTestServer.run start zk server on port:35609
[junit4] 2> 72345 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72389 T37 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1af289f name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72389 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72391 T23 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 72412 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72413 T39 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2305a name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72414 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72418 T23 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 72424 T23 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 72429 T23 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 72432 T23 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 72438 T23 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 72445 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 72452 T23 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 72452 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 72456 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
[junit4] 2> 72457 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
[junit4] 2> 72457 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
[junit4] 2> 72457 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
[junit4] 2> 72458 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
[junit4] 2> 72458 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
[junit4] 2> 72458 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
[junit4] 2> 72459 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
[junit4] 2> 72459 T23 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
[junit4] 2> 72463 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 72467 T41 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cd7e13 name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 72468 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 72690 T23 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 73021 T23 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 73079 T23 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:33655
[junit4] 2> 73141 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 73142 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 73142 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659
[junit4] 2> 73143 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/'
[junit4] 2> 73163 T23 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/solr.xml
[junit4] 2> 73193 T23 oasc.CoreContainer.<init> New CoreContainer 26862557
[junit4] 2> 73194 T23 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/]
[junit4] 2> 73195 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 73196 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 73196 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 73197 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 73198 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 73200 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 73201 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 73202 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 73202 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 73226 T23 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 73226 T23 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 73227 T23 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 73228 T23 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35609/solr
[junit4] 2> 73248 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 73258 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 73269 T53 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f8c512 name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 73274 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 73294 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 73308 T55 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@503b85 name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 73308 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 73315 T23 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 73323 T23 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 73345 T23 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 73363 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33655_
[junit4] 2> 73366 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33655_
[junit4] 2> 73371 T23 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 73375 T23 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 73386 T23 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:33655_
[junit4] 2> 73387 T23 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 73390 T23 oasc.Overseer.start Overseer (id=91316112551903236-127.0.0.1:33655_-n_0000000000) starting
[junit4] 2> 73450 T23 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 73494 T57 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 73495 T23 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 73495 T56 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 73502 T57 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 73508 T23 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 73510 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 73528 T58 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 73529 T58 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 73540 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 73540 T58 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 73544 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 73546 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33655",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33655_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 73547 T56 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 73558 T56 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 73565 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 73571 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 74541 T58 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 74541 T58 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/collection1
[junit4] 2> 74541 T58 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 74542 T58 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 74543 T58 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 74544 T58 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 74545 T58 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/collection1/'
[junit4] 2> 74580 T58 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 74590 T58 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 74597 T58 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 74615 T58 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 74653 T58 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 74659 T58 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableLongField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 74667 T58 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableFloatField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 74671 T58 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableDoubleField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 74895 T58 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 74900 T58 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 74904 T58 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 74904 T58 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 74909 T58 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 74910 T58 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1393373299659/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/
[junit4] 2> 74910 T58 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 74911 T58 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data
[junit4] 2> 74912 T58 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index/
[junit4] 2> 74912 T58 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 74913 T58 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index
[junit4] 2> 74915 T58 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1a928d1 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 74915 T58 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74917 T58 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 74917 T58 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 74918 T58 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 74918 T58 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 74924 T58 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 74925 T58 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 74925 T58 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 74925 T58 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 74928 T58 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 74931 T58 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 74936 T58 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 74937 T58 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1a928d1 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/control/data/index),segFN=segments_1,generation=1}
[junit4] 2> 74938 T58 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 74938 T58 oass.SolrIndexSearcher.<init> Opening Searcher@17e7fc3[collection1] main
[junit4] 2> 74946 T59 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17e7fc3[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 74947 T58 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 74953 T62 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:33655 collection:control_collection shard:shard1
[junit4] 2> 74953 T23 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 74954 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 74964 T62 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 74983 T62 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 74989 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 74990 T62 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 74990 T62 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@498617 url=https://127.0.0.1:33655/collection1 node=127.0.0.1:33655_ C2_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:33655, core=collection1, node_name=127.0.0.1:33655_}
[junit4] 2> 74992 T62 C2 P33655 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:33655/collection1/
[junit4] 2> 74993 T62 C2 P33655 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 74993 T62 C2 P33655 oasc.SyncStrategy.syncToMe https://127.0.0.1:33655/collection1/ has no replicas
[junit4] 2> 74993 T62 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:33655/collection1/ shard1
[junit4] 2> 74994 T62 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 74997 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 75015 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 75018 T64 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ecdae name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 75020 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 75022 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 75026 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75033 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75035 T23 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 75068 T23 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 75072 T23 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 75077 T23 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 75081 T23 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:59298
[junit4] 2> 75083 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 75084 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 75084 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224
[junit4] 2> 75085 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/'
[junit4] 2> 75099 T23 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/solr.xml
[junit4] 2> 75116 T23 oasc.CoreContainer.<init> New CoreContainer 17678257
[junit4] 2> 75116 T23 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/]
[junit4] 2> 75117 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 75117 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 75117 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 75118 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 75118 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 75118 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 75119 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 75119 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 75119 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 75123 T23 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 75124 T23 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 75124 T23 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 75125 T23 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35609/solr
[junit4] 2> 75125 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 75126 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 75152 T75 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55f571 name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 75153 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 75154 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75155 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 75155 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 75162 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 75167 T77 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1eb0b7b name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 75167 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 75181 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 75186 T62 oasc.ZkController.register We are https://127.0.0.1:33655/collection1/ and leader is https://127.0.0.1:33655/collection1/
[junit4] 2> 75187 T62 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:33655
[junit4] 2> 75187 T62 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 75187 T62 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 75187 T62 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 75189 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75190 T62 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 75192 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 75193 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:33655",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33655_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 75197 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 75300 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 75300 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 75300 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 76186 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59298_
[junit4] 2> 76187 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59298_
[junit4] 2> 76189 T77 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 76189 T64 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 76190 T55 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 76199 T78 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 76200 T78 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 76202 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76202 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76202 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76202 T78 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 76203 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 76204 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:59298",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:59298_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 76205 T56 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 76205 T56 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 76208 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 76314 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 76314 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 76314 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77206 T78 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 77206 T78 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/collection1
[junit4] 2> 77206 T78 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 77207 T78 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 77207 T78 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 77209 T78 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 77209 T78 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/collection1/'
[junit4] 2> 77240 T78 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 77268 T78 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 77272 T78 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 77290 T78 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 77503 T78 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 77505 T78 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 77507 T78 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 77507 T78 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 77508 T78 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 77508 T78 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1393373302224/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/
[junit4] 2> 77509 T78 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 77510 T78 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1
[junit4] 2> 77510 T78 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index/
[junit4] 2> 77510 T78 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 77510 T78 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index
[junit4] 2> 77512 T78 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4a20a1 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 77512 T78 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 77513 T78 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 77513 T78 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 77513 T78 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 77513 T78 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 77513 T78 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 77516 T78 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 77516 T78 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 77516 T78 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 77518 T78 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 77520 T78 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 77520 T78 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 77520 T78 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4a20a1 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty1/index),segFN=segments_1,generation=1}
[junit4] 2> 77521 T78 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 77521 T78 oass.SolrIndexSearcher.<init> Opening Searcher@1780fa7[collection1] main
[junit4] 2> 77522 T79 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1780fa7[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 77522 T78 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 77523 T23 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 77523 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 77523 T82 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:59298 collection:collection1 shard:shard2
[junit4] 2> 77525 T82 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 77531 T82 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 77533 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77533 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77533 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77534 T82 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 77534 T82 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@16dc16 url=https://127.0.0.1:59298/collection1 node=127.0.0.1:59298_ C3_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:59298, core=collection1, node_name=127.0.0.1:59298_}
[junit4] 2> 77535 T82 C3 P59298 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:59298/collection1/
[junit4] 2> 77535 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77536 T82 C3 P59298 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 77536 T82 C3 P59298 oasc.SyncStrategy.syncToMe https://127.0.0.1:59298/collection1/ has no replicas
[junit4] 2> 77536 T82 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:59298/collection1/ shard2
[junit4] 2> 77536 T82 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 77541 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77548 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77551 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77560 T23 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 77561 T23 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 77565 T23 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 77570 T23 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:54886
[junit4] 2> 77572 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 77572 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 77573 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713
[junit4] 2> 77573 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/'
[junit4] 2> 77584 T23 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/solr.xml
[junit4] 2> 77599 T23 oasc.CoreContainer.<init> New CoreContainer 12936508
[junit4] 2> 77603 T23 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/]
[junit4] 2> 77604 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 77604 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 77605 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 77605 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 77606 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 77606 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 77607 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 77607 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 77607 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 77618 T23 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 77618 T23 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 77619 T23 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 77619 T23 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35609/solr
[junit4] 2> 77620 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 77621 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77623 T93 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12fa9c0 name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77624 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77626 T32 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446b8542ed0008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:744)
[junit4] 2>
[junit4] 2> 77627 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 77640 T95 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@169237c name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 77640 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 77649 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 77655 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77655 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77655 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77655 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77699 T82 oasc.ZkController.register We are https://127.0.0.1:59298/collection1/ and leader is https://127.0.0.1:59298/collection1/
[junit4] 2> 77699 T82 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:59298
[junit4] 2> 77700 T82 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 77700 T82 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 77700 T82 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 77701 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77701 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77701 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77702 T82 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77703 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 77704 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:59298",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:59298_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 77707 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 77809 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77809 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77810 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 77810 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 78653 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54886_
[junit4] 2> 78655 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54886_
[junit4] 2> 78657 T64 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 78657 T95 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 78657 T55 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 78657 T77 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 78664 T96 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 78665 T96 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 78666 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78666 T96 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 78666 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78666 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78668 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 78668 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:54886",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54886_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 78669 T56 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 78669 T56 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 78671 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 78774 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78774 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78774 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 78774 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 79666 T96 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 79667 T96 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/collection1
[junit4] 2> 79667 T96 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 79668 T96 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 79668 T96 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 79669 T96 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 79669 T96 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/collection1/'
[junit4] 2> 79688 T96 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 79699 T96 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 79700 T96 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 79713 T96 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 79845 T96 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 79847 T96 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 79850 T96 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 79850 T96 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 79851 T96 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 79852 T96 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1393373304713/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/
[junit4] 2> 79852 T96 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 79853 T96 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2
[junit4] 2> 79854 T96 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index/
[junit4] 2> 79854 T96 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 79854 T96 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index
[junit4] 2> 79857 T96 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1ca5b09 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 79857 T96 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79860 T96 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 79861 T96 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 79862 T96 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 79862 T96 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 79862 T96 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 79863 T96 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 79863 T96 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 79863 T96 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 79866 T96 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 79867 T96 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 79868 T96 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 79869 T96 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1ca5b09 lockFactory=NativeFSLockFactory@./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty2/index),segFN=segments_1,generation=1}
[junit4] 2> 79869 T96 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 79870 T96 oass.SolrIndexSearcher.<init> Opening Searcher@1f739c4[collection1] main
[junit4] 2> 79871 T96 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 79872 T97 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f739c4[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 79872 T23 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0
[junit4] 2> 79872 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 79874 T100 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:54886 collection:collection1 shard:shard1
[junit4] 2> 79876 T100 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 79891 T100 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 79894 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79894 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79894 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79894 T100 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 79895 T100 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@125d3d8 url=https://127.0.0.1:54886/collection1 node=127.0.0.1:54886_ C4_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:54886, core=collection1, node_name=127.0.0.1:54886_}
[junit4] 2> 79895 T100 C4 P54886 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:54886/collection1/
[junit4] 2> 79895 T100 C4 P54886 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 79895 T100 C4 P54886 oasc.SyncStrategy.syncToMe https://127.0.0.1:54886/collection1/ has no replicas
[junit4] 2> 79896 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 79896 T100 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:54886/collection1/ shard1
[junit4] 2> 79896 T100 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 79900 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79903 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79907 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 79917 T23 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 79918 T23 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 79927 T23 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 79932 T23 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:56893
[junit4] 2> 79935 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 79935 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 79936 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063
[junit4] 2> 79936 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/'
[junit4] 2> 79959 T23 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J0/./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/solr.xml
[junit4] 2> 79981 T23 oasc.CoreContainer.<init> New CoreContainer 9700943
[junit4] 2> 79982 T23 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/]
[junit4] 2> 79982 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 79983 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 79983 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 79984 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 79984 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 79985 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 79985 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 79986 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 79986 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 79993 T23 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 79993 T23 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 79994 T23 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 79994 T23 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35609/solr
[junit4] 2> 79995 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 80004 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 80007 T111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35bb65 name:ZooKeeperConnection Watcher:127.0.0.1:35609 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 80008 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 80012 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80012 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80013 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80013 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80015 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 80020 T113 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@189522c name:ZooKeeperConnection Watcher:127.0.0.1:35609/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 80020 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 80026 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 80054 T100 oasc.ZkController.register We are https://127.0.0.1:54886/collection1/ and leader is https://127.0.0.1:54886/collection1/
[junit4] 2> 80055 T100 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:54886
[junit4] 2> 80055 T100 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 80055 T100 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 80055 T100 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 80057 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80058 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80058 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80058 T100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 80060 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 80061 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:54886",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54886_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 80064 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 80167 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80167 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80167 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80167 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 80167 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 81030 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56893_
[junit4] 2> 81031 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56893_
[junit4] 2> 81034 T95 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81034 T55 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81035 T77 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81034 T64 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81034 T113 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 81045 T114 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 81045 T114 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 81046 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81046 T114 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 81047 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81047 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81048 T56 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 81049 T56 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:56893",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:56893_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 81049 T56 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 81050 T56 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 81052 T55 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 81155 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 81155 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 81155 T77 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 81155 T64 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 81155 T55 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 82047 T114 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 82047 T114 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/collection1
[junit4] 2> 82047 T114 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 82048 T114 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 82048 T114 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 82049 T114 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 82050 T114 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/collection1/'
[junit4] 2> 82074 T114 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 82088 T114 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 82089 T114 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 82100 T114 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 82283 T114 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 82284 T114 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 82286 T114 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 82286 T114 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 82286 T114 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 82287 T114 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1393373307063/collection1/, dataDir=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty3/
[junit4] 2> 82287 T114 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 82288 T114 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty3
[junit4] 2> 82288 T114 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty3/index/
[junit4] 2> 82288 T114 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 82288 T114 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.client.solrj.impl.CloudSolrServerTest-1393373299429
[...truncated too long message...]
te params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 13
[junit4] 2> 303513 T82 C34 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=1
[junit4] 2> 303515 T81 C34 oasc.SolrCore.execute [collection1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 303516 T81 C34 oasc.SolrCore.execute [collection1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 303516 T74 oas.SolrTestCaseJ4.tearDown ###Ending testPingHandler
[junit4] 2> 303539 T74 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=28609177
[junit4] 2> 303540 T74 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@16b4852
[junit4] 2> 303540 T74 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=62,autocommit maxTime=15000ms,autocommits=1,soft autocommits=3,optimizes=2,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=1,cumulative_adds=248,cumulative_deletesById=5,cumulative_deletesByQuery=30,cumulative_errors=1,transaction_logs_total_size=1243,transaction_logs_total_number=10}
[junit4] 2> 303541 T74 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 303541 T74 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 303541 T74 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 303542 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: now flush at close waitForMerges=true
[junit4] 2> 303542 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: start flush: applyAllDeletes=true
[junit4] 2> 303542 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: index before flush
[junit4] 2> 303542 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker startFullFlush
[junit4] 2> 303543 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 303543 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker finishFullFlush success=true
[junit4] 2> 303543 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: apply all deletes during flush
[junit4] 2> 303544 T74 C34 oasu.LoggingInfoStream.message [BD][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: prune sis=org.apache.lucene.index.SegmentInfos@bedfe minGen=9223372036854775807 packetCount=0
[junit4] 2> 303544 T74 C34 oasu.LoggingInfoStream.message [CMS][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: now merge
[junit4] 2> 303545 T74 C34 oasu.LoggingInfoStream.message [CMS][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: index:
[junit4] 2> 303545 T74 C34 oasu.LoggingInfoStream.message [CMS][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: no more merges pending; now return
[junit4] 2> 303545 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: waitForMerges
[junit4] 2> 303545 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: waitForMerges done
[junit4] 2> 303546 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: now call final commit()
[junit4] 2> 303546 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: start
[junit4] 2> 303546 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: enter lock
[junit4] 2> 303547 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: now prepare
[junit4] 2> 303547 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: prepareCommit: flush
[junit4] 2> 303548 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: index before flush
[junit4] 2> 303548 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker startFullFlush
[junit4] 2> 303548 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 303549 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: apply all deletes during flush
[junit4] 2> 303549 T74 C34 oasu.LoggingInfoStream.message [BD][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: prune sis=org.apache.lucene.index.SegmentInfos@bedfe minGen=9223372036854775807 packetCount=0
[junit4] 2> 303549 T74 C34 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker finishFullFlush success=true
[junit4] 2> 303549 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: startCommit(): start
[junit4] 2> 303550 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: skip startCommit(): no changes pending
[junit4] 2> 303550 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: pendingCommit == null; skip
[junit4] 2> 303550 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: commit: done
[junit4] 2> 303550 T74 C34 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleBinaryTest-seed#[CA32FB715F20BCA7]-worker]: at close:
[junit4] 2> 303551 T74 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 303552 T74 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 303552 T74 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760;done=false>>]
[junit4] 2> 303552 T74 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760
[junit4] 2> 303552 T74 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760/index;done=false>>]
[junit4] 2> 303553 T74 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/solrtest-SolrExampleBinaryTest-1393373322760/index
[junit4] 2> 303553 T74 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4] 2> 303605 T74 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=sr_ME, timezone=US/East-Indiana
[junit4] 2> NOTE: Linux 3.8.0-36-generic i386/Oracle Corporation 1.8.0 (32-bit)/cpus=8,threads=1,free=32826296,total=64880640
[junit4] 2> NOTE: All tests run in this JVM: [TestNamedListCodec, SolrQueryTest, MergeIndexesEmbeddedTest, SolrExampleStreamingTest, SolrDocumentTest, SolrExampleBinaryTest]
[junit4] Completed on J1 in 208.08s, 26 tests, 1 error <<< FAILURES!
[...truncated 23 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:465: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:445: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1274: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 48 suites, 280 tests, 2 errors
Total time: 56 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-fcs-b129 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure