You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2020/06/16 18:21:30 UTC
[JENKINS] Lucene-Solr-master-Windows (64bit/jdk-12.0.2) - Build #
8984 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/8984/
Java: 64bit/jdk-12.0.2 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.OverseerTest.testShardLeaderChange
Error Message:
Captured an uncaught exception in thread: Thread[id=15116, name=OverseerCollectionConfigSetProcessor-72093689740328991-127.0.0.1:59744_solr-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15116, name=OverseerCollectionConfigSetProcessor-72093689740328991-127.0.0.1:59744_solr-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]
at __randomizedtesting.SeedInfo.seed([35B84E9715708BA4:EBEBC9600FE87E55]:0)
Caused by: org.apache.solr.common.AlreadyClosedException
at __randomizedtesting.SeedInfo.seed([35B84E9715708BA4]:0)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:69)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:435)
at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:155)
at java.base/java.lang.Thread.run(Thread.java:835)
Build Log:
[...truncated 13814 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> 1015161 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of 'C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\server\solr\configsets\_default\conf'
[junit4] 2> 1015162 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.SolrTestCaseJ4 Created dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J3\temp\solr.cloud.OverseerTest_35B84E9715708BA4-001\data-dir-142-001
[junit4] 2> 1015163 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1015163 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 1015163 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1015163 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1015163 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
[junit4] 2> 1015163 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1015264 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port: 59744
[junit4] 2> 1015264 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:59744
[junit4] 2> 1015264 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:59744
[junit4] 2> 1015264 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 59744
[junit4] 2> 1015268 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1015273 INFO (zkConnectionManagerCallback-9441-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1015273 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1015277 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1015281 INFO (zkConnectionManagerCallback-9443-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1015281 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1015281 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 1015281 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 1015293 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testReplay
[junit4] 2> 1015410 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1015419 INFO (zkConnectionManagerCallback-9447-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1015419 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1015425 WARN (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5e3c8456[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1015425 WARN (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5e3c8456[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1015426 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1015427 WARN (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2516ee27[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1015427 WARN (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2516ee27[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1015511 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1015516 INFO (zkConnectionManagerCallback-9456-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1015516 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1015522 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1015526 INFO (zkConnectionManagerCallback-9461-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1015526 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1015526 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1015535 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59744_solr
[junit4] 2> 1015535 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328962-127.0.0.1:59744_solr-n_0000000000) starting
[junit4] 2> 1015561 WARN (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState.
[junit4] 2> 1015569 INFO (OverseerStateUpdate-72093689740328962-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59744_solr
[junit4] 2> 1015591 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1015607 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1015615 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1015732 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1016052 INFO (closeThreadPool-9467-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72093689740328962-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1016052 INFO (OverseerStateUpdate-72093689740328962-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59744_solr
[junit4] 2> 1016052 INFO (OverseerAutoScalingTriggerThread-72093689740328962-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1016055 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper
[junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:398) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1016143 INFO (TEST-OverseerTest.testReplay-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testReplay
[junit4] 2> 1016152 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOverseerFailure
[junit4] 2> 1016340 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1016346 INFO (zkConnectionManagerCallback-9471-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016346 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1016361 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1016362 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1016363 INFO (zkCallback-9470-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1016372 INFO (zkConnectionManagerCallback-9475-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016372 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1016382 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4ca91fa5[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1016382 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4ca91fa5[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1016382 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1016385 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@12026c8c[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1016385 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@12026c8c[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1016387 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1016392 INFO (zkConnectionManagerCallback-9484-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016392 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1016397 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1016451 INFO (zkConnectionManagerCallback-9489-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016451 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1016451 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1016456 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1016462 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59744_solr
[junit4] 2> 1016463 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328966-127.0.0.1:59744_solr-n_0000000000) starting
[junit4] 2> 1016477 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState.
[junit4] 2> 1016482 INFO (OverseerStateUpdate-72093689740328966-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59744_solr
[junit4] 2> 1016489 INFO (OverseerStateUpdate-72093689740328966-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1016492 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1016495 INFO (zkConnectionManagerCallback-9498-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016495 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1016497 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1016498 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1016605 INFO (zkCallback-9470-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1016613 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
[junit4] 2> 1016616 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72093689740328965-node1_core1-n_0000000000
[junit4] 2> 1016623 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1
[junit4] 2> 1016632 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
[junit4] 2> 1016633 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72093689740328965-node1_core1-n_0000000001
[junit4] 2> 1016636 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1
[junit4] 2> 1016740 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1016741 INFO (zkCallback-9442-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1016741 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1016751 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
[junit4] 2> 1016753 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72093689740328965-node1_core1-n_0000000002
[junit4] 2> 1016858 WARN (OverseerStateUpdate-72093689740328966-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Solr cannot talk to ZK, exiting Overseer main queue loop
[junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:2021) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:315) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:315) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.writePendingUpdates(ZkStateWriter.java:181) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:324) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1016858 INFO (OverseerStateUpdate-72093689740328966-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59744_solr
[junit4] 2> 1016860 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1016863 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper
[junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:398) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1016863 INFO (zkConnectionManagerCallback-9500-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016863 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1016863 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328966-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1016863 INFO (OverseerAutoScalingTriggerThread-72093689740328966-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1016872 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@58da60e3[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1016872 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@58da60e3[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1016872 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1016874 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2ef0ab97[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1016874 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2ef0ab97[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1016876 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1016879 INFO (zkConnectionManagerCallback-9509-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016879 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1016884 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1016886 INFO (zkConnectionManagerCallback-9514-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016887 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1016888 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1016888 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1016892 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59744_solr
[junit4] 2> 1016892 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328970-127.0.0.1:59744_solr-n_0000000001) starting
[junit4] 2> 1016895 WARN (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState.
[junit4] 2> 1016895 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1
[junit4] 2> 1016899 INFO (OverseerStateUpdate-72093689740328970-127.0.0.1:59744_solr-n_0000000001) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59744_solr
[junit4] 2> 1016903 INFO (OverseerStateUpdate-72093689740328970-127.0.0.1:59744_solr-n_0000000001) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1016923 INFO (zkCallback-9442-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1016927 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1016927 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1017047 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1017047 INFO (zkCallback-9442-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1017047 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1017047 INFO (zkCallback-9442-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1017050 INFO (zkCallback-9488-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1017050 INFO (zkCallback-9499-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1017050 INFO (zkCallback-9470-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1017050 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1017051 INFO (zkCallback-9513-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1017051 INFO (zkCallback-9497-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1017769 INFO (closeThreadPool-9520-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72093689740328966-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1017769 INFO (closeThreadPool-9520-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72093689740328970-127.0.0.1:59744_solr-n_0000000001) closing
[junit4] 2> 1017769 INFO (OverseerStateUpdate-72093689740328970-127.0.0.1:59744_solr-n_0000000001) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59744_solr
[junit4] 2> 1017770 INFO (OverseerAutoScalingTriggerThread-72093689740328970-127.0.0.1:59744_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1017772 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper
[junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:398) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1017813 INFO (TEST-OverseerTest.testOverseerFailure-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testOverseerFailure
[junit4] 2> 1017819 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testExternalClusterStateChangeBehavior
[junit4] 2> 1017913 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1017915 INFO (zkConnectionManagerCallback-9524-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1017915 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1017918 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5481b707[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1017918 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5481b707[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1017919 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1017920 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@10d2b187[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1017920 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@10d2b187[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1017921 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1017923 INFO (zkConnectionManagerCallback-9533-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1017923 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1017926 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1017928 INFO (zkConnectionManagerCallback-9538-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1017928 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1017930 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1017932 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59744_solr
[junit4] 2> 1017933 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328973-127.0.0.1:59744_solr-n_0000000000) starting
[junit4] 2> 1017939 WARN (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState.
[junit4] 2> 1017942 INFO (OverseerStateUpdate-72093689740328973-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59744_solr
[junit4] 2> 1017947 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1017949 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (c1) shard=shard1 coreNodeName=core_node1
[junit4] 2> 1018050 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1018050 INFO (zkCallback-9442-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1018050 INFO (zkCallback-9442-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1018155 INFO (zkCallback-9442-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/test/state.json] for collection [test] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1018165 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (test) shard=x coreNodeName=core_node1
[junit4] 2> 1018166 INFO (OverseerStateUpdate-72093689740328973-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"test",
[junit4] 2> "shard":"x",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "state":"down"}
[junit4] 2> 1018268 ERROR (OverseerStateUpdate-72093689740328973-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Exception in Overseer main queue loop
[junit4] 2> => org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /collections/test/state.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:122)
[junit4] 2> org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /collections/test/state.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:122) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:2384) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$setData$6(SolrZkClient.java:351) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.setData(SolrZkClient.java:351) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.writePendingUpdates(ZkStateWriter.java:185) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:324) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1018274 INFO (zkCallback-9442-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test/state.json] for collection [test] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1018275 INFO (OverseerStateUpdate-72093689740328973-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"test",
[junit4] 2> "shard":"x",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "state":"down"}
[junit4] 2> 1018275 INFO (zkCallback-9442-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test/state.json] for collection [test] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1018277 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (c1) shard=shard1 coreNodeName=core_node1
[junit4] 2> 1018586 INFO (closeThreadPool-9544-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72093689740328973-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1018586 INFO (OverseerStateUpdate-72093689740328973-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59744_solr
[junit4] 2> 1018586 INFO (OverseerAutoScalingTriggerThread-72093689740328973-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1018589 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper
[junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:398) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1018649 INFO (TEST-OverseerTest.testExternalClusterStateChangeBehavior-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testExternalClusterStateChangeBehavior
[junit4] 2> 1018657 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBadQueueItem
[junit4] 2> 1018803 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1018805 INFO (zkConnectionManagerCallback-9546-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1018806 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1018810 WARN (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5b7b9541[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1018810 WARN (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5b7b9541[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1018811 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1018811 WARN (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@34c35bdd[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1018811 WARN (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@34c35bdd[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1018811 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1018817 INFO (zkConnectionManagerCallback-9555-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1018817 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1018823 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1018824 INFO (zkConnectionManagerCallback-9560-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1018824 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1018827 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1018831 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59744_solr
[junit4] 2> 1018831 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328976-127.0.0.1:59744_solr-n_0000000000) starting
[junit4] 2> 1018843 WARN (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState.
[junit4] 2> 1018848 INFO (OverseerStateUpdate-72093689740328976-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59744_solr
[junit4] 2> 1018853 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1018862 INFO (zkConnectionManagerCallback-9567-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1018862 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1018880 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1018887 INFO (zkCallback-9545-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1018891 INFO (zkCallback-9566-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1018893 INFO (zkCallback-9559-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1018896 INFO (zkCallback-9566-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019000 INFO (zkCallback-9566-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019007 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
[junit4] 2> 1019010 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72093689740328979-127.0.0.1_core1-n_0000000000
[junit4] 2> 1019117 INFO (zkCallback-9566-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019117 INFO (zkCallback-9566-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019120 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2
[junit4] 2> 1019122 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard2/leader after winning as /collections/collection1/leader_elect/shard2/election/72093689740328979-127.0.0.1_core2-n_0000000000
[junit4] 2> 1019230 INFO (zkCallback-9566-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019230 INFO (zkCallback-9566-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019235 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard3
[junit4] 2> 1019239 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard3/leader after winning as /collections/collection1/leader_elect/shard3/election/72093689740328979-127.0.0.1_core3-n_0000000000
[junit4] 2> 1019346 INFO (zkCallback-9442-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019346 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019346 INFO (zkCallback-9442-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019346 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019351 ERROR (OverseerStateUpdate-72093689740328976-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has no collection specified '{
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":"",
[junit4] 2> "shard":"shard1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}'
[junit4] 2> 1019353 ERROR (OverseerStateUpdate-72093689740328976-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has no collection specified '{
[junit4] 2> "operation":"deletecore",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":""}'
[junit4] 2> 1019462 INFO (zkCallback-9566-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019467 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection2/leaders/shard1
[junit4] 2> 1019469 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection2/leaders/shard1/leader after winning as /collections/collection2/leader_elect/shard1/election/72093689740328979-127.0.0.1_core1-n_0000000000
[junit4] 2> 1019576 INFO (zkCallback-9566-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019576 INFO (zkCallback-9566-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019579 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection2/leaders/shard2
[junit4] 2> 1019581 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection2/leaders/shard2/leader after winning as /collections/collection2/leader_elect/shard2/election/72093689740328979-127.0.0.1_core2-n_0000000000
[junit4] 2> 1019689 INFO (zkCallback-9566-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019689 INFO (zkCallback-9566-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019692 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection2/leaders/shard3
[junit4] 2> 1019695 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection2/leaders/shard3/leader after winning as /collections/collection2/leader_elect/shard3/election/72093689740328979-127.0.0.1_core3-n_0000000000
[junit4] 2> 1019800 INFO (zkCallback-9442-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019800 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019800 INFO (zkCallback-9442-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019800 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019805 INFO (zkCallback-9559-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1019805 INFO (zkCallback-9442-thread-4) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1019805 INFO (zkCallback-9566-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1019805 INFO (zkCallback-9545-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1020218 INFO (closeThreadPool-9572-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72093689740328976-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1020218 INFO (OverseerAutoScalingTriggerThread-72093689740328976-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1020218 INFO (OverseerStateUpdate-72093689740328976-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59744_solr
[junit4] 2> 1020221 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper
[junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:398) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1020266 INFO (TEST-OverseerTest.testBadQueueItem-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBadQueueItem
[junit4] 2> 1020273 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDoubleAssignment
[junit4] 2> 1020354 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1020357 INFO (zkConnectionManagerCallback-9576-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1020357 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1020362 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020362 INFO (zkCallback-9575-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020363 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1020365 INFO (zkConnectionManagerCallback-9580-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1020365 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1020368 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@236c08ab[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1020368 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@236c08ab[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1020368 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1020369 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@54bf31ad[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1020369 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@54bf31ad[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1020371 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1020373 INFO (zkConnectionManagerCallback-9589-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1020373 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1020375 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1020377 INFO (zkConnectionManagerCallback-9594-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1020377 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1020379 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020379 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1020381 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59744_solr
[junit4] 2> 1020382 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328981-127.0.0.1:59744_solr-n_0000000000) starting
[junit4] 2> 1020388 WARN (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState.
[junit4] 2> 1020390 INFO (OverseerStateUpdate-72093689740328981-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59744_solr
[junit4] 2> 1020391 INFO (OverseerStateUpdate-72093689740328981-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020393 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1020396 INFO (zkConnectionManagerCallback-9603-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1020396 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1020397 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020398 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1020502 INFO (zkCallback-9575-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1020504 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
[junit4] 2> 1020506 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72093689740328980-node1_core1-n_0000000000
[junit4] 2> 1020511 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1
[junit4] 2> 1020515 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1020515 INFO (zkCallback-9575-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1020515 INFO (zkCallback-9593-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1020515 INFO (zkCallback-9579-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1020515 INFO (zkCallback-9602-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1020617 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1020619 INFO (zkConnectionManagerCallback-9605-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1020619 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1020624 INFO (zkCallback-9579-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020624 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020624 INFO (zkCallback-9593-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020624 INFO (zkCallback-9602-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020625 INFO (zkCallback-9604-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1020628 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
[junit4] 2> 1020629 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72093689740328985-node1_core1-n_0000000001
[junit4] 2> 1020733 WARN (OverseerStateUpdate-72093689740328981-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Solr cannot talk to ZK, exiting Overseer main queue loop
[junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:2021) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:315) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:315) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.overseer.ZkStateWriter.writePendingUpdates(ZkStateWriter.java:181) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:324) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1020733 INFO (OverseerStateUpdate-72093689740328981-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59744_solr
[junit4] 2> 1020738 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper
[junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:398) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1020740 INFO (zkCallback-9593-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1020740 INFO (zkCallback-9604-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1020740 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1020740 INFO (zkCallback-9602-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1021149 INFO (closeThreadPool-9609-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72093689740328981-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1021149 INFO (OverseerAutoScalingTriggerThread-72093689740328981-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1021183 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDoubleAssignment
[junit4] 2> 1021189 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOverseerStatsReset
[junit4] 2> 1021271 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1021274 INFO (zkConnectionManagerCallback-9613-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1021274 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1021279 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1021280 INFO (zkCallback-9612-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1021282 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1021284 INFO (zkConnectionManagerCallback-9620-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1021284 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1021285 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1021285 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1021288 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@556d1709[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1021288 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@556d1709[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1021288 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1021289 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@50dbd049[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1021289 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@50dbd049[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1021292 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59744_solr
[junit4] 2> 1021292 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328961-127.0.0.1:59744_solr-n_0000000000) starting
[junit4] 2> 1021298 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState.
[junit4] 2> 1021302 INFO (OverseerStateUpdate-72093689740328961-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59744_solr
[junit4] 2> 1021306 INFO (zkCallback-9612-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1021411 INFO (zkCallback-9612-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1021418 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
[junit4] 2> 1021420 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72093689740328986-node1_core1-n_0000000000
[junit4] 2> 1021422 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328961-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1021423 INFO (OverseerAutoScalingTriggerThread-72093689740328961-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1021423 INFO (OverseerStateUpdate-72093689740328961-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59744_solr
[junit4] 2> 1021425 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328961-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1021425 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328961-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1021426 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328961-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1021426 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=72093689740328961-127.0.0.1:59744_solr-n_0000000000) am exiting, but I'm still the leader
[junit4] 2> 1021427 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59744_solr
[junit4] 2> 1021427 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328961-127.0.0.1:59744_solr-n_0000000001) starting
[junit4] 2> 1021432 WARN (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState.
[junit4] 2> 1021435 INFO (zkCallback-9612-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1021435 INFO (zkCallback-9619-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1021436 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1021438 INFO (OverseerStateUpdate-72093689740328961-127.0.0.1:59744_solr-n_0000000001) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59744_solr
[junit4] 2> 1021640 INFO (closeThreadPool-9635-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72093689740328961-127.0.0.1:59744_solr-n_0000000001) closing
[junit4] 2> 1021640 INFO (OverseerStateUpdate-72093689740328961-127.0.0.1:59744_solr-n_0000000001) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59744_solr
[junit4] 2> 1021640 INFO (OverseerAutoScalingTriggerThread-72093689740328961-127.0.0.1:59744_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1021642 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I (id=72093689740328961-127.0.0.1:59744_solr-n_0000000001) am exiting, but I'm still the leader
[junit4] 2> 1021676 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testOverseerStatsReset
[junit4] 2> 1021682 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testStateChange
[junit4] 2> 1021763 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1021765 INFO (zkConnectionManagerCallback-9639-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1021765 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1021768 WARN (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@63a5fb65[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1021768 WARN (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@63a5fb65[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1021768 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1021770 WARN (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@27fb16cc[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1021770 WARN (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@27fb16cc[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1021771 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1021773 INFO (zkConnectionManagerCallback-9648-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1021773 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1021775 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1021777 INFO (zkConnectionManagerCallback-9653-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1021777 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1021779 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59744/solr ready
[junit4] 2> 1021782 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59744_solr
[junit4] 2> 1021782 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Overseer (id=72093689740328988-127.0.0.1:59744_solr-n_0000000000) starting
[junit4] 2> 1021789 WARN (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.Overseer Unable to check back-compat of .system collection - can't obtain ClusterState.
[junit4] 2> 1021791 INFO (OverseerStateUpdate-72093689740328988-127.0.0.1:59744_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59744_solr
[junit4] 2> 1021794 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1021794 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1
[junit4] 2> 1021898 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1021898 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1021898 INFO (zkCallback-9442-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1021899 INFO (TEST-OverseerTest.testStateChange-seed#[35B84E9715708BA4]) [ ] o.a.s.c.AbstractDistribZkTestBase verifyReplicaStatus (collection1) shard=shard1 coreNodeName=core_node1
[junit4] 2> 1022002 INFO (zkCallback-9442-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1022002 INFO (zkCallback-9442-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1022002 INFO (zkCallback-9442-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1022002 INFO (zkCallback-9442-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1022211 INFO (closeThreadPool-9659-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72093689740328988-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1022211 INFO (OverseerStateUpdate-72093689740328988-127.0.0.1:59744_solr-n_0000000000) [ ]
[...truncated too long message...]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection4/state.json] for collection [collection4] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1030472 INFO (zkCallback-9953-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1030472 INFO (zkCallback-9969-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1030472 INFO (zkCallback-9442-thread-8) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1030472 INFO (zkCallback-9955-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1030472 INFO (zkCallback-9938-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1031086 INFO (closeThreadPool-9976-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72093689740329035-127.0.0.1:59744_solr-n_0000000000) closing
[junit4] 2> 1031086 INFO (closeThreadPool-9976-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72093689740329040-127.0.0.1:59744_solr-n_0000000001) closing
[junit4] 2> 1031086 INFO (OverseerStateUpdate-72093689740329040-127.0.0.1:59744_solr-n_0000000001) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59744_solr
[junit4] 2> 1031086 INFO (OverseerAutoScalingTriggerThread-72093689740329040-127.0.0.1:59744_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1031088 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer Error communicating with ZooKeeper
[junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131) ~[zookeeper-3.5.7.jar:3.5.7]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:398) ~[java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 1031139 INFO (TEST-OverseerTest.testDownNodeFailover-seed#[35B84E9715708BA4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDownNodeFailover
[junit4] 1> / (0)
[junit4] 1>
[junit4] 2> 1031141 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
[junit4] 2> 1031345 WARN (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 70 /solr/aliases.json
[junit4] 2> 70 /solr/clusterprops.json
[junit4] 2> 7 /solr/collections/collection1/state.json
[junit4] 2> 2 /solr/collections/collection2/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 179 /solr/collections/collection1/state.json
[junit4] 2> 20 /solr/autoscaling.json
[junit4] 2> 18 /solr/collections/collection2/state.json
[junit4] 2> 8 /solr/collections/c1/state.json
[junit4] 2> 5 /solr/collections/test/state.json
[junit4] 2> 5 /solr/collections/collection0/state.json
[junit4] 2> 3 /solr/collections/collection4/state.json
[junit4] 2> 2 /solr/collections/collection3/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 277 /solr/live_nodes
[junit4] 2> 177 /solr/overseer/queue
[junit4] 2> 129 /solr/collections
[junit4] 2> 22 /solr/overseer/queue-work
[junit4] 2> 20 /solr/autoscaling/events/.scheduled_maintenance
[junit4] 2> 20 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 19 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> 1031355 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:59744
[junit4] 2> 1031355 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:59744
[junit4] 2> 1031355 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 59744
[junit4] 2> 1033362 INFO (SUITE-OverseerTest-seed#[35B84E9715708BA4]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFieldsData(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=7, maxDocsPerChunk=4, blockShift=15), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=7, blockSize=15)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@5ade3fd), locale=af, timezone=America/Paramaribo
[junit4] 2> NOTE: Windows 10 10.0 amd64/AdoptOpenJDK 12.0.2 (64-bit)/cpus=6,threads=1,free=343519464,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestDelegationWithHadoopAuth, HdfsBasicDistributedZkTest, CollectionTooManyReplicasTest, TestReload, SolrLogPostToolTest, HLLSerializationTest, AddReplicaTest, TestApiFramework, SplitShardTest, TestGroupingSearch, ExternalFileFieldSortTest, SpellCheckCollatorWithCollapseTest, SolrIndexSearcherTest, TestHdfsBackupRestoreCore, DistributedQueryComponentCustomSortTest, DeleteNodeTest, TestTlogReplayVsRecovery, TestCloudJSONFacetSKG, SolrMetricManagerTest, RoutingToNodesWithPropertiesTest, SolrIndexMetricsTest, TestFieldCacheReopen, DOMUtilTest, BasicAuthIntegrationTest, IndexSizeTriggerTest, TestExactSharedStatsCache, TestWaitForStateWithJettyShutdowns, TestReversedWildcardFilterFactory, TestSurroundQueryParser, XCJFQueryTest, OutputWriterTest, ZookeeperStatusHandlerTest, TestPseudoReturnFields, TestBM25SimilarityFactory, CursorPagingTest, TestSolrFieldCacheBean, DirectSolrConnectionTest, RandomizedTaggerTest, TestSolrIndexConfig, LeaderFailureAfterFreshStartTest, TestSmileRequest, TestFaceting, TestExactStatsCacheCloud, CategoryRoutedAliasUpdateProcessorTest, TestRequestStatusCollectionAPI, MultiSolrCloudTestCaseTest, ReplaceNodeNoTargetTest, OverriddenZkACLAndCredentialsProvidersTest, VersionInfoTest, SolrLogAuditLoggerPluginTest, DateRangeFieldTest, NumberUtilsTest, MaxSizeAutoCommitTest, ScheduledTriggerIntegrationTest, TestInPlaceUpdateWithRouteField, FastVectorHighlighterTest, TestDocSet, TestManagedSchema, ChaosMonkeyNothingIsSafeTest, ZkDynamicConfigTest, TestSimPolicyCloud, ResourceLoaderTest, TestBadConfig, ForceLeaderTest, CollectionPropsTest, UniqFieldsUpdateProcessorFactoryTest, XsltUpdateRequestHandlerTest, PeerSyncWithLeaderTest, TestPolicyCloud, TestFieldCollectionResource, TestSegmentSorting, TestSchemaResource, TestCloudPseudoReturnFields, SuggesterFSTTest, TestPrepRecovery, CachingDirectoryFactoryTest, TestSimpleTrackingShardHandler, ZkStateReaderTest, TestNestedUpdateProcessor, TestJavabinTupleStreamParser, TestEmbeddedSolrServerConstructors, HdfsDirectoryFactoryTest, PreAnalyzedUpdateProcessorTest, HdfsUnloadDistributedZkTest, TestPartialUpdateDeduplication, TestLMDirichletSimilarityFactory, TestStressThreadBackup, TestCollapseQParserPlugin, SolrInfoBeanTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedGrouping, TestSolrTestCaseJ4, TestTolerantSearch, TestTrie, PathHierarchyTokenizerFactoryTest, ProtectedTermFilterFactoryTest, ChaosMonkeySafeLeaderTest, ChaosMonkeySafeLeaderWithPullReplicasTest, ConnectionManagerTest, CreateCollectionCleanupTest, DeleteShardTest, DeleteStatusTest, DistribCursorPagingTest, DistribJoinFromCollectionTest, FullSolrCloudDistribCmdsTest, HttpPartitionOnCommitTest, HttpPartitionWithTlogReplicasTest, OverseerModifyCollectionTest, OverseerRolesTest, OverseerTest]
[junit4] Completed [482/910 (1!)] on J3 in 18.22s, 14 tests, 1 error, 1 skipped <<< FAILURES!
[...truncated 53509 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/8984/consoleText
[repro] Revision: 87a3bef50f8c08404ee8bd66ca868caf5dd072cb
[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC"
[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean
[...truncated 6 lines...]
[repro] Test suites by module:
[repro] solr\core
[repro] OverseerTest
[repro] ant compile-test
[...truncated 2473 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.OverseerTest" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Dtests.seed=35B84E9715708BA4 -Dtests.slow=true -Dtests.locale=af -Dtests.timezone=America/Paramaribo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[...truncated 142 lines...]
[repro] Failures w/original seeds:
[repro] 0/5 failed: org.apache.solr.cloud.OverseerTest
[repro] Exiting with code 0
[...truncated 78 lines...]