You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2015/01/06 02:41:08 UTC
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 2038
- Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/2038/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC (asserts: false)
2 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([C377F6CDB370922C]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([C377F6CDB370922C]:0)
Build Log:
[...truncated 9652 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/init-core-data-001
[junit4] 2> 2473018 T12481 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 2473019 T12481 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_/k
[junit4] 2> 2473031 T12481 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 2473032 T12481 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2473033 T12482 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 2473134 T12481 oasc.ZkTestServer.run start zk server on port:52374
[junit4] 2> 2473135 T12481 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2473137 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2473148 T12489 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37ae381c name:ZooKeeperConnection Watcher:127.0.0.1:52374 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2473149 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2473150 T12481 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2473151 T12481 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2473162 T12481 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2473163 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2473166 T12492 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@717ce765 name:ZooKeeperConnection Watcher:127.0.0.1:52374/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2473166 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2473167 T12481 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2473167 T12481 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 2473176 T12481 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 2473185 T12481 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 2473191 T12481 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 2473199 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2473200 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 2473211 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 2473212 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 2473220 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2473221 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2473227 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2473227 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 2473235 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2473236 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 2473243 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2473244 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 2473252 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2473253 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 2473265 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2473265 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 2473275 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2473276 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2473285 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2473286 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 2473295 T12481 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2473296 T12481 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 2473311 T12481 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2473313 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2473317 T12495 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a06c979 name:ZooKeeperConnection Watcher:127.0.0.1:52374/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2473317 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2473318 T12481 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2475353 T12481 oejs.Server.doStart jetty-9.2.6.v20141205
[junit4] 2> 2475361 T12481 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@125cdf40{/_/k,null,AVAILABLE}
[junit4] 2> 2475376 T12481 oejs.AbstractConnector.doStart Started ServerConnector@5d1694f6{SSL-http/1.1}{127.0.0.1:52378}
[junit4] 2> 2475377 T12481 oejs.Server.doStart Started @2479032ms
[junit4] 2> 2475380 T12481 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4554617c
[junit4] 2> 2475381 T12481 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2475381 T12481 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-002
[junit4] 2> 2475381 T12481 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-002/'
[junit4] 2> 2475428 T12481 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-002/solr.xml
[junit4] 2> 2475441 T12481 oasc.CoreContainer.<init> New CoreContainer 1477053068
[junit4] 2> 2475441 T12481 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-002/]
[junit4] 2> 2475442 T12481 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2475442 T12481 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2475443 T12481 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2475443 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2475443 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 2475444 T12481 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2475444 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2475444 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2475445 T12481 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2475445 T12481 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2475447 T12481 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 2475449 T12481 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2475450 T12481 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2475450 T12481 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2475451 T12481 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52374/solr
[junit4] 2> 2475451 T12481 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2475452 T12481 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2475454 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2475459 T12509 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6237f021 name:ZooKeeperConnection Watcher:127.0.0.1:52374 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2475460 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2475460 T12481 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2475465 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2475470 T12512 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c2a353b name:ZooKeeperConnection Watcher:127.0.0.1:52374/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2475470 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2475474 T12481 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2475483 T12481 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 2475493 T12481 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 2475499 T12481 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 2475506 T12481 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 2475515 T12481 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 2475518 T12481 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52378__%2Fk
[junit4] 2> 2475519 T12481 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52378__%2Fk
[junit4] 2> 2475527 T12481 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 2475532 T12481 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 2475537 T12481 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2475542 T12481 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52378__%2Fk
[junit4] 2> 2475542 T12481 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2475548 T12481 oasc.Overseer.start Overseer (id=93093957939691524-127.0.0.1:52378__%2Fk-n_0000000000) starting
[junit4] 2> 2475557 T12481 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 2475577 T12481 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 2475578 T12514 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2475580 T12481 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 2475584 T12481 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 2475588 T12481 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2475599 T12513 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2475602 T12516 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 2475603 T12516 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2475604 T12516 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2475604 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2475612 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52378/_/k",
[junit4] 2> "node_name":"127.0.0.1:52378__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2475615 T12513 oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52378/_/k",
[junit4] 2> "node_name":"127.0.0.1:52378__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 2475615 T12513 oasco.ClusterStateMutator.createCollection building a new cName: control_collection
[junit4] 2> 2475615 T12513 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2475619 T12512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2476606 T12516 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2476606 T12516 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 2476609 T12516 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2476610 T12516 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 2476615 T12516 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2476616 T12516 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-002/collection1/'
[junit4] 2> 2476617 T12516 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20C377F6CDB370922C-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 2476618 T12516 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20C377F6CDB370922C-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 2476619 T12516 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20C377F6CDB370922C-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 2476650 T12516 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 2476656 T12516 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 2476667 T12516 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 2476669 T12516 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
[junit4] 2> 2476679 T12516 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2476684 T12516 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 2476687 T12516 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2476788 T12516 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2476789 T12516 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2476791 T12516 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2476799 T12516 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2476863 T12516 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 2476863 T12516 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2476864 T12516 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-002/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/control/data/
[junit4] 2> 2476864 T12516 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@35281535
[junit4] 2> 2476869 T12516 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/control/data
[junit4] 2> 2476870 T12516 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/control/data/index/
[junit4] 2> 2476871 T12516 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 2476874 T12516 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/control/data/index
[junit4] 2> 2476875 T12516 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=5.9208984375, floorSegmentMB=1.71875, forceMergeDeletesPctAllowed=27.90470388285157, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2476880 T12516 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 2476881 T12516 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2476888 T12516 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2476888 T12516 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2476889 T12516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2476889 T12516 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2476889 T12516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2476889 T12516 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2476890 T12516 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2476891 T12516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2476891 T12516 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2476892 T12516 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2476892 T12516 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2476892 T12516 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2476893 T12516 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2476894 T12516 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 2476895 T12516 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 2476896 T12516 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 2476896 T12516 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 2476896 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 2476897 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 2476898 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 2476900 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 2476900 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 2476901 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 2476901 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 2476902 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 2476902 T12516 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2476903 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2476903 T12516 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2476911 T12516 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2476912 T12516 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2476913 T12516 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2476914 T12516 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2476915 T12516 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 2476916 T12516 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2476916 T12516 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2476917 T12516 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2476919 T12516 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 2476919 T12516 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2476919 T12516 oass.SolrIndexSearcher.<init> Opening Searcher@e3a743b[collection1] main
[junit4] 2> 2476919 T12516 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 2476922 T12516 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2476922 T12516 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 2476923 T12516 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2476923 T12516 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 2476924 T12516 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2476925 T12516 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 2476925 T12516 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2476926 T12516 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2476926 T12516 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 2476927 T12516 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2476928 T12517 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e3a743b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2476929 T12520 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:52378/_/k collection:control_collection shard:shard1
[junit4] 2> 2476930 T12481 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 2476930 T12481 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2476931 T12520 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 2476936 T12481 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2476938 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2476942 T12523 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3018da02 name:ZooKeeperConnection Watcher:127.0.0.1:52374/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2476943 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2476943 T12481 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2476946 T12520 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2476946 T12481 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2476950 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2476952 T12520 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2476952 T12520 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2849 name=collection1 org.apache.solr.core.SolrCore@3435a001 url=https://127.0.0.1:52378/_/k/collection1 node=127.0.0.1:52378__%2Fk C2849_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:52378/_/k, node_name=127.0.0.1:52378__%2Fk, state=down}
[junit4] 2> 2476952 T12520 C2849 P52378 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:52378/_/k/collection1/
[junit4] 2> 2476953 T12520 C2849 P52378 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2476953 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state version: 1
[junit4] 2> 2476953 T12520 C2849 P52378 oasc.SyncStrategy.syncToMe https://127.0.0.1:52378/_/k/collection1/ has no replicas
[junit4] 2> 2476954 T12520 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:52378/_/k/collection1/ shard1
[junit4] 2> 2476955 T12520 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 2476959 T12481 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2476959 T12481 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
[junit4] 2> 2476959 T12481 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2476961 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2476964 T12526 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b336fd name:ZooKeeperConnection Watcher:127.0.0.1:52374/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2476964 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2476965 T12481 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2476966 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2476970 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"https://127.0.0.1:52378/_/k",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 2476982 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "name":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "stateFormat":2} current state version: 1
[junit4] 2> 2476982 T12513 oasco.ClusterStateMutator.createCollection building a new cName: collection1
[junit4] 2> 2476984 T12512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2476986 T12523 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2477015 T12527 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2477018 T12513 oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/collection1/state.json
[junit4] 2> 2477022 T12523 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2477073 T12520 oasc.ZkController.register We are https://127.0.0.1:52378/_/k/collection1/ and leader is https://127.0.0.1:52378/_/k/collection1/
[junit4] 2> 2477073 T12520 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52378/_/k
[junit4] 2> 2477073 T12520 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2477074 T12520 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 2477074 T12520 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2477076 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2477082 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52378/_/k",
[junit4] 2> "node_name":"127.0.0.1:52378__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 3
[junit4] 2> 2477085 T12513 oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52378/_/k",
[junit4] 2> "node_name":"127.0.0.1:52378__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 2477192 T12523 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2477192 T12512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2478871 T12481 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 2478873 T12481 oejs.Server.doStart jetty-9.2.6.v20141205
[junit4] 2> 2478877 T12481 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@3def848e{/_/k,null,AVAILABLE}
[junit4] 2> 2478883 T12481 oejs.AbstractConnector.doStart Started ServerConnector@3ce3cd36{SSL-http/1.1}{127.0.0.1:52383}
[junit4] 2> 2478884 T12481 oejs.Server.doStart Started @2482539ms
[junit4] 2> 2478887 T12481 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4554617c
[junit4] 2> 2478888 T12481 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2478888 T12481 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-003
[junit4] 2> 2478889 T12481 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-003/'
[junit4] 2> 2478949 T12481 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-003/solr.xml
[junit4] 2> 2478965 T12481 oasc.CoreContainer.<init> New CoreContainer 1944658852
[junit4] 2> 2478965 T12481 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-003/]
[junit4] 2> 2478967 T12481 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2478968 T12481 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2478969 T12481 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2478969 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2478969 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 2478969 T12481 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2478970 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2478970 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2478971 T12481 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2478971 T12481 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2478973 T12481 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 2478976 T12481 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2478976 T12481 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2478976 T12481 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2478976 T12481 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52374/solr
[junit4] 2> 2478978 T12481 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2478978 T12481 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2478980 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2478983 T12541 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@317f8465 name:ZooKeeperConnection Watcher:127.0.0.1:52374 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2478983 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2478983 T12481 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2478992 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2478992 T12544 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d03fc21 name:ZooKeeperConnection Watcher:127.0.0.1:52374/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2478993 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2479007 T12481 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2480025 T12481 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52383__%2Fk
[junit4] 2> 2480030 T12481 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52383__%2Fk
[junit4] 2> 2480047 T12481 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2480078 T12545 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 2480079 T12545 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2480083 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2480085 T12545 oasc.ZkController.preRegister Registering watch for external collection collection1
[junit4] 2> 2480085 T12545 oascc.ZkStateReader.addZkWatch addZkWatch collection1
[junit4] 2> 2480085 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52383/_/k",
[junit4] 2> "node_name":"127.0.0.1:52383__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 2480086 T12545 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/collection1/state.json from ZooKeeper...
[junit4] 2> 2480088 T12513 oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52383/_/k",
[junit4] 2> "node_name":"127.0.0.1:52383__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 2480088 T12513 oasco.ReplicaMutator.updateState Collection already exists with numShards=1
[junit4] 2> 2480088 T12513 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2480090 T12545 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 0
[junit4] 2> 2480090 T12545 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2480092 T12513 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 0
[junit4] 2> 2480093 T12544 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 2480095 T12544 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 1
[junit4] 2> 2481092 T12545 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2481092 T12545 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2481096 T12545 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2481096 T12545 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2481102 T12545 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2481103 T12545 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-003/collection1/'
[junit4] 2> 2481107 T12545 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20C377F6CDB370922C-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 2481108 T12545 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20C377F6CDB370922C-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 2481109 T12545 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20C377F6CDB370922C-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 2481136 T12545 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 2481142 T12545 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 2481153 T12545 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 2481156 T12545 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
[junit4] 2> 2481189 T12545 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2481192 T12545 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 2481198 T12545 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2481281 T12545 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2481283 T12545 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2481284 T12545 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2481288 T12545 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2481357 T12545 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2481357 T12545 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2481358 T12545 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-003/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty1/
[junit4] 2> 2481358 T12545 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@35281535
[junit4] 2> 2481360 T12545 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty1
[junit4] 2> 2481361 T12545 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty1/index/
[junit4] 2> 2481362 T12545 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 2481363 T12545 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty1/index
[junit4] 2> 2481364 T12545 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=5.9208984375, floorSegmentMB=1.71875, forceMergeDeletesPctAllowed=27.90470388285157, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2481368 T12545 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 2481368 T12545 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2481383 T12545 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2481384 T12545 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2481384 T12545 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2481385 T12545 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2481385 T12545 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2481386 T12545 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2481387 T12545 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2481387 T12545 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2481388 T12545 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2481389 T12545 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2481389 T12545 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2481390 T12545 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2481390 T12545 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2481391 T12545 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 2481391 T12545 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 2481393 T12545 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 2481394 T12545 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 2481394 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 2481396 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 2481396 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 2481397 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 2481398 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 2481398 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 2481399 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 2481400 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 2481401 T12545 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2481402 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2481402 T12545 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2481409 T12545 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2481410 T12545 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2481412 T12545 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2481414 T12545 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2481414 T12545 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 2481417 T12545 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2481417 T12545 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2481419 T12545 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2481420 T12545 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 2481421 T12545 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2481421 T12545 oass.SolrIndexSearcher.<init> Opening Searcher@4e7bbea0[collection1] main
[junit4] 2> 2481422 T12545 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2481429 T12545 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2481429 T12545 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 2481431 T12545 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2481431 T12545 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 2481431 T12545 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2481433 T12545 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 2481433 T12545 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2481434 T12545 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2481434 T12545 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 2481435 T12546 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e7bbea0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2481436 T12545 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2481438 T12549 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:52383/_/k collection:collection1 shard:shard1
[junit4] 2> 2481439 T12481 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 2481439 T12481 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2481440 T12549 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2481473 T12549 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2481489 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2481492 T12549 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2481493 T12549 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2850 name=collection1 org.apache.solr.core.SolrCore@7ad1b530 url=https://127.0.0.1:52383/_/k/collection1 node=127.0.0.1:52383__%2Fk C2850_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:52383/_/k, node_name=127.0.0.1:52383__%2Fk, state=down}
[junit4] 2> 2481493 T12549 C2850 P52383 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:52383/_/k/collection1/
[junit4] 2> 2481494 T12549 C2850 P52383 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2481494 T12549 C2850 P52383 oasc.SyncStrategy.syncToMe https://127.0.0.1:52383/_/k/collection1/ has no replicas
[junit4] 2> 2481495 T12549 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:52383/_/k/collection1/ shard1
[junit4] 2> 2481496 T12549 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2481497 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1"} current state version: 4
[junit4] 2> 2481503 T12513 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 1
[junit4] 2> 2481508 T12544 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 2481514 T12544 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 2
[junit4] 2> 2481540 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2481544 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:52383/_/k",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 4
[junit4] 2> 2481547 T12513 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 2
[junit4] 2> 2481554 T12544 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 2481560 T12544 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 3
[junit4] 2> 2481598 T12549 oasc.ZkController.register We are https://127.0.0.1:52383/_/k/collection1/ and leader is https://127.0.0.1:52383/_/k/collection1/
[junit4] 2> 2481599 T12549 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52383/_/k
[junit4] 2> 2481599 T12549 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2481599 T12549 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 2481600 T12549 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2481603 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2481607 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52383/_/k",
[junit4] 2> "node_name":"127.0.0.1:52383__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 2481609 T12513 oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52383/_/k",
[junit4] 2> "node_name":"127.0.0.1:52383__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 2481612 T12513 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 3
[junit4] 2> 2481614 T12544 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 2481616 T12549 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4
[junit4] 2> 2481620 T12544 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4
[junit4] 2> 2483508 T12481 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 2483509 T12481 oejs.Server.doStart jetty-9.2.6.v20141205
[junit4] 2> 2483513 T12481 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@6f19e9d0{/_/k,null,AVAILABLE}
[junit4] 2> 2483534 T12481 oejs.AbstractConnector.doStart Started ServerConnector@5b3e7b3a{SSL-http/1.1}{127.0.0.1:52386}
[junit4] 2> 2483534 T12481 oejs.Server.doStart Started @2487189ms
[junit4] 2> 2483544 T12481 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4554617c
[junit4] 2> 2483544 T12481 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2483544 T12481 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-004
[junit4] 2> 2483545 T12481 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-004/'
[junit4] 2> 2483583 T12481 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-004/solr.xml
[junit4] 2> 2483617 T12481 oasc.CoreContainer.<init> New CoreContainer 1595985892
[junit4] 2> 2483617 T12481 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-004/]
[junit4] 2> 2483618 T12481 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2483619 T12481 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2483619 T12481 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2483619 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2483620 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 2483620 T12481 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2483620 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2483621 T12481 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2483621 T12481 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2483622 T12481 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2483640 T12481 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 2483642 T12481 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2483642 T12481 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2483642 T12481 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2483643 T12481 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52374/solr
[junit4] 2> 2483643 T12481 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2483643 T12481 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2483645 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2483649 T12563 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5950401b name:ZooKeeperConnection Watcher:127.0.0.1:52374 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2483649 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2483650 T12481 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2483660 T12481 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2483661 T12566 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2848d55 name:ZooKeeperConnection Watcher:127.0.0.1:52374/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2483664 T12481 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2483689 T12481 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2484706 T12481 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52386__%2Fk
[junit4] 2> 2484710 T12481 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52386__%2Fk
[junit4] 2> 2484727 T12481 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2484745 T12567 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 2484745 T12567 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2484751 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2484753 T12567 oasc.ZkController.preRegister Registering watch for external collection collection1
[junit4] 2> 2484753 T12567 oascc.ZkStateReader.addZkWatch addZkWatch collection1
[junit4] 2> 2484754 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52386/_/k",
[junit4] 2> "node_name":"127.0.0.1:52386__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 2484755 T12567 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/collection1/state.json from ZooKeeper...
[junit4] 2> 2484759 T12513 oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52386/_/k",
[junit4] 2> "node_name":"127.0.0.1:52386__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 2484759 T12513 oasco.ReplicaMutator.updateState Collection already exists with numShards=1
[junit4] 2> 2484759 T12513 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2484761 T12567 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4
[junit4] 2> 2484761 T12567 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2484762 T12513 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 4
[junit4] 2> 2484764 T12544 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 2484764 T12566 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 2484765 T12544 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 5
[junit4] 2> 2484765 T12566 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 5
[junit4] 2> 2485763 T12567 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2485764 T12567 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2485766 T12567 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2485766 T12567 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2485774 T12567 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2485774 T12567 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-004/collection1/'
[junit4] 2> 2485775 T12567 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20C377F6CDB370922C-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 2485776 T12567 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20C377F6CDB370922C-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 2485777 T12567 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20C377F6CDB370922C-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 2485808 T12567 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 2485812 T12567 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 2485822 T12567 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 2485823 T12567 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
[junit4] 2> 2485834 T12567 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2485838 T12567 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 2485842 T12567 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2485936 T12567 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2485938 T12567 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2485940 T12567 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2485945 T12567 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2486004 T12567 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2486004 T12567 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2486005 T12567 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-004/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty2/
[junit4] 2> 2486006 T12567 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@35281535
[junit4] 2> 2486007 T12567 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty2
[junit4] 2> 2486008 T12567 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty2/index/
[junit4] 2> 2486009 T12567 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 2486010 T12567 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty2/index
[junit4] 2> 2486010 T12567 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=5.9208984375, floorSegmentMB=1.71875, forceMergeDeletesPctAllowed=27.90470388285157, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2486016 T12567 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 2486017 T12567 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2486021 T12567 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2486022 T12567 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2486022 T12567 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2486023 T12567 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2486023 T12567 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2486024 T12567 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2486024 T12567 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2486025 T12567 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2486026 T12567 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2486029 T12567 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2486030 T12567 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2486031 T12567 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2486031 T12567 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2486032 T12567 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 2486034 T12567 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 2486036 T12567 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 2486037 T12567 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 2486037 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 2486042 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 2486043 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 2486043 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 2486044 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 2486045 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 2486046 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 2486046 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 2486047 T12567 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2486048 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2486048 T12567 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2486057 T12567 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2486058 T12567 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2486059 T12567 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2486061 T12567 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2486061 T12567 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 2486064 T12567 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2486064 T12567 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2486065 T12567 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2486068 T12567 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest C377F6CDB370922C-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 2486068 T12567 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2486071 T12567 oass.SolrIndexSearcher.<init> Opening Searcher@1d19f8d8[collection1] main
[junit4] 2> 2486071 T12567 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2486075 T12567 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2486075 T12567 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 2486077 T12567 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2486077 T12567 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 2486077 T12567 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2486078 T12567 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 2486079 T12567 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2486079 T12567 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2486079 T12567 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 2486080 T12567 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2486081 T12568 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d19f8d8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2486082 T12571 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:52386/_/k collection:collection1 shard:shard1
[junit4] 2> 2486085 T12481 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 2486085 T12481 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2486102 T12481 oasc.AbstractFullDistribZkTestBase.enableAutoSoftCommit Turning on auto soft commit: 1000
[junit4] 2> 2486120 T12571 oasc.ZkController.register We are https://127.0.0.1:52386/_/k/collection1/ and leader is https://127.0.0.1:52383/_/k/collection1/
[junit4] 2> 2486120 T12571 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52386/_/k
[junit4] 2> 2486120 T12571 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 2486122 T12571 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C2851 name=collection1 org.apache.solr.core.SolrCore@7f75c9d3 url=https://127.0.0.1:52386/_/k/collection1 node=127.0.0.1:52386__%2Fk C2851_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:52386/_/k, node_name=127.0.0.1:52386__%2Fk, state=down}
[junit4] 2> 2486123 T12573 C2851 P52386 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 2486123 T12573 C2851 P52386 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 2486125 T12573 C2851 P52386 oasc.RecoveryStrategy.doRecovery Publishing state of core collection1 as recovering, leader is https://127.0.0.1:52383/_/k/collection1/ and I am https://127.0.0.1:52386/_/k/collection1/
[junit4] 2> 2486125 T12573 C2851 P52386 oasc.ZkController.publish publishing core=collection1 state=recovering collection=collection1
[junit4] 2> 2486125 T12573 C2851 P52386 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2486130 T12512 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2486132 T12513 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52386/_/k",
[junit4] 2> "node_name":"127.0.0.1:52386__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"recovering",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 2486140 T12573 C2851 P52386 oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to https://127.0.0.1:52383/_/k; WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1%3A52386__%252Fk&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
[junit4] 2> 2486153 T12513 oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52386/_/k",
[junit4] 2> "node_name":"127.0.0.1:52386__%2Fk",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"recovering",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 2486164 T12513 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 5
[junit4] 2> 2486166 T12566 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 2486169 T12544 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeData
[...truncated too long message...]
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene50): {rnd_b=PostingsFormat(name=Memory doPackFST= false), _version_=FST50, a_t=PostingsFormat(name=LuceneVarGapFixedInterval), a_i=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=Memory doPackFST= false)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=de_LU, timezone=Europe/Prague
[junit4] 2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_20 (64-bit)/cpus=2,threads=17,free=247279576,total=392167424
[junit4] 2> NOTE: All tests run in this JVM: [AddBlockUpdateTest, DisMaxRequestHandlerTest, TestSortingResponseWriter, TestSolrDeletionPolicy1, TestQueryUtils, TestShortCircuitedRequests, TermsComponentTest, TestSolrQueryParser, TestRemoteStreaming, DistributedQueryComponentOptimizationTest, TestRealTimeGet, HighlighterConfigTest, TestSolrXmlPersistor, OverseerCollectionProcessorTest, LeaderElectionIntegrationTest, SuggesterWFSTTest, JSONWriterTest, SSLMigrationTest, AssignTest, TestPerFieldSimilarity, StressHdfsTest, TestBinaryResponseWriter, DocExpirationUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, CursorMarkTest, TestBadConfig, CloudExitableDirectoryReaderTest, PathHierarchyTokenizerFactoryTest, TestFileDictionaryLookup, TestManagedResourceStorage, BadCopyFieldTest, TestUpdate, TestMissingGroups, DocValuesTest, SuggesterTest, SpellCheckComponentTest, CachingDirectoryFactoryTest, SolrCoreTest, TestCloudPivotFacet, TestReload, TestZkChroot, FullSolrCloudDistribCmdsTest, TestInitQParser, TestQuerySenderListener, DistanceFunctionTest, SolrIndexSplitterTest, SortByFunctionTest, SuggesterFSTTest, CollectionsAPIDistributedZkTest, TestSchemaSimilarityResource, LeaderInitiatedRecoveryOnCommitTest, TestSearchPerf, TestComponentsName, DistributedExpandComponentTest, TestLFUCache, CurrencyFieldXmlFileTest, TestJmxIntegration, TestRequestStatusCollectionAPI, TestCustomSort, SyncSliceTest, DistributedFacetPivotLongTailTest, ZkControllerTest, TestDefaultSimilarityFactory, PluginInfoTest, OverseerTest, TestDFRSimilarityFactory, TimeZoneUtilsTest, HdfsBasicDistributedZkTest, TestValueSourceCache, TestStandardQParsers, JsonLoaderTest, TestIndexSearcher, BadComponentTest, DistribCursorPagingTest, TestSolr4Spatial2, CustomCollectionTest, AutoCommitTest, TestReRankQParserPlugin, SolrRequestParserTest, ExternalCollectionsTest, TestPartialUpdateDeduplication, VMParamsZkACLAndCredentialsProvidersTest, TestDistributedGrouping, PreAnalyzedUpdateProcessorTest, DistributedDebugComponentTest, BasicDistributedZkTest, TestShardHandlerFactory, DeleteInactiveReplicaTest, HdfsCollectionsAPIDistributedZkTest, TestFoldingMultitermQuery, TestFieldSortValues, DeleteShardTest, TestXIncludeConfig, DistanceUnitsTest, TestModifyConfFiles, SpellingQueryConverterTest, FieldAnalysisRequestHandlerTest, TestNoOpRegenerator, TestBulkSchemaConcurrent, TestCSVLoader, TestSolrXml, TestSimpleQParserPlugin, SampleTest, TestUniqueKeyFieldResource, TestReplicaProperties, SpellPossibilityIteratorTest, TestWordDelimiterFilterFactory, ExternalFileFieldSortTest, OutputWriterTest, SpellCheckCollatorTest, DateRangeFieldTest, TestManagedSchemaFieldTypeResource, StatsComponentTest, TestOrdValues, MigrateRouteKeyTest, ChangedSchemaMergeTest, TestCloudSchemaless, HttpPartitionTest, TestSystemIdResolver, TestLRUCache, CurrencyFieldOpenExchangeTest, NumericFieldsTest, TestIndexingPerformance, DistributedTermsComponentTest, HighlighterTest, TestReplicationHandler, SpatialRPTFieldTypeTest, ZkCLITest, TestComplexPhraseQParserPlugin, TestAnalyzeInfixSuggestions, SuggestComponentTest, TestPseudoReturnFields, TestCSVResponseWriter, OpenCloseCoreStressTest, ClusterStateTest, TestDistributedMissingSort, SpatialFilterTest, HdfsBasicDistributedZk2Test, TestSolr4Spatial, TestAtomicUpdateErrorCases, TestStressUserVersions, TestCopyFieldCollectionResource, HardAutoCommitTest, AnalysisAfterCoreReloadTest, FieldMutatingUpdateProcessorTest, TestCollationFieldDocValues, DirectSolrConnectionTest, RecoveryZkTest, FileUtilsTest, TestSchemaVersionResource, TestCharFilters, TestSolrQueryParserDefaultOperatorResource, TestDistribDocBasedVersion, ShardRoutingTest, TestExceedMaxTermLength, PingRequestHandlerTest, DistributedIntervalFacetingTest, TestReloadAndDeleteDocs, BasicDistributedZk2Test, TestBlendedInfixSuggestions, TestSweetSpotSimilarityFactory, SimpleMLTQParserTest, ShowFileRequestHandlerTest, TestSchemaManager, DateFieldTest, SolrIndexConfigTest, TestCollapseQParserPlugin, QueryEqualityTest, MinimalSchemaTest, TestDefaultStatsCache, PolyFieldTest, ResponseLogComponentTest, TestQueryTypes, DistributedFacetPivotWhiteBoxTest, RankQueryTest, HdfsRecoveryZkTest, TestExtendedDismaxParser, PeerSyncTest, TestSolrCoreProperties, TestStressLucene, TestRandomFaceting, SynonymTokenizerTest, TestFieldTypeCollectionResource, FacetPivotSmallTest, TestDocBasedVersionConstraints, TestFaceting, DistributedQueueTest, TestRTGBase, PrimUtilsTest, TestJoin, HdfsSyncSliceTest, AnalysisErrorHandlingTest, HdfsLockFactoryTest, TestSurroundQueryParser, TestSolrXmlPersistence, TestCoreDiscovery, TestIntervalFaceting, TestBinaryField, EnumFieldTest, TermVectorComponentDistributedTest, TestDynamicLoading, OverseerStatusTest, StandardRequestHandlerTest, MergeStrategyTest, ParsingFieldUpdateProcessorsTest, TestManagedSchemaFieldResource, SliceStateTest, SimpleFacetsTest, TestExpandComponent, URLClassifyProcessorTest, TestDefaultSearchFieldResource, TestNonNRTOpen, SoftAutoCommitTest, TestDynamicFieldCollectionResource, ReturnFieldsTest, TestSolrDynamicMBean, EchoParamsTest, DocValuesMissingTest, TestFieldTypeResource, TestJmxMonitoredMap, SimpleCollectionCreateDeleteTest, TestConfig, ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=C377F6CDB370922C -Dtests.slow=true -Dtests.locale=de_LU -Dtests.timezone=Europe/Prague -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([C377F6CDB370922C]:0)
[junit4] Completed in 7224.12s, 1 test, 2 errors <<< FAILURES!
[...truncated 693 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:519: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:467: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:61: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:187: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:510: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1348: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:956: There were test failures: 462 suites, 1869 tests, 1 suite-level error, 1 error, 736 ignored (21 assumptions)
Total time: 199 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any