You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/02/20 04:58:28 UTC
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 452 -
Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/452/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 8 in http://127.0.0.1:63485/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 8 in http://127.0.0.1:63485/solr
at __randomizedtesting.SeedInfo.seed([B34AFF72C3CCB2C9:72BA86DEEE9C786E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:885)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:599)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13973 lines...]
[junit4] Suite: org.apache.solr.cloud.TestTlogReplica
[junit4] 2> 3402430 INFO (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/init-core-data-001
[junit4] 2> 3402430 WARN (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=73 numCloses=73
[junit4] 2> 3402430 INFO (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 3402431 INFO (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 3402432 INFO (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001
[junit4] 2> 3402432 INFO (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3402432 INFO (Thread-10386) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3402432 INFO (Thread-10386) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3402433 ERROR (Thread-10386) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3402536 INFO (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:54360
[junit4] 2> 3402538 INFO (zkConnectionManagerCallback-6539-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3402543 INFO (jetty-launcher-6536-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3402543 INFO (jetty-launcher-6536-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3402544 INFO (jetty-launcher-6536-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3402544 INFO (jetty-launcher-6536-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3402545 INFO (jetty-launcher-6536-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 3402545 INFO (jetty-launcher-6536-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@8098841{/solr,null,AVAILABLE}
[junit4] 2> 3402546 INFO (jetty-launcher-6536-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3402546 INFO (jetty-launcher-6536-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3402546 INFO (jetty-launcher-6536-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 3402546 INFO (jetty-launcher-6536-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7acc0df8{HTTP/1.1,[http/1.1]}{127.0.0.1:63485}
[junit4] 2> 3402546 INFO (jetty-launcher-6536-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@55b33511{/solr,null,AVAILABLE}
[junit4] 2> 3402546 INFO (jetty-launcher-6536-thread-1) [ ] o.e.j.s.Server Started @3409668ms
[junit4] 2> 3402546 INFO (jetty-launcher-6536-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63485}
[junit4] 2> 3402546 INFO (jetty-launcher-6536-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@e6d975f{HTTP/1.1,[http/1.1]}{127.0.0.1:51759}
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-2) [ ] o.e.j.s.Server Started @3409668ms
[junit4] 2> 3402547 ERROR (jetty-launcher-6536-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51759}
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3402547 ERROR (jetty-launcher-6536-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-20T04:20:59.021Z
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3402547 INFO (jetty-launcher-6536-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-20T04:20:59.021Z
[junit4] 2> 3402549 INFO (zkConnectionManagerCallback-6541-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3402549 INFO (zkConnectionManagerCallback-6543-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3402550 INFO (jetty-launcher-6536-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3402550 INFO (jetty-launcher-6536-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3402567 INFO (jetty-launcher-6536-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54360/solr
[junit4] 2> 3402568 INFO (jetty-launcher-6536-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54360/solr
[junit4] 2> 3402569 INFO (zkConnectionManagerCallback-6548-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3402570 INFO (zkConnectionManagerCallback-6551-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3402572 INFO (zkConnectionManagerCallback-6553-thread-1-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3402573 INFO (zkConnectionManagerCallback-6555-thread-1-processing-n:127.0.0.1:63485_solr) [n:127.0.0.1:63485_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3402704 INFO (jetty-launcher-6536-thread-2) [n:127.0.0.1:51759_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3402705 INFO (jetty-launcher-6536-thread-2) [n:127.0.0.1:51759_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51759_solr
[junit4] 2> 3402706 INFO (jetty-launcher-6536-thread-1) [n:127.0.0.1:63485_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3402706 INFO (jetty-launcher-6536-thread-2) [n:127.0.0.1:51759_solr ] o.a.s.c.Overseer Overseer (id=72229147587379205-127.0.0.1:51759_solr-n_0000000000) starting
[junit4] 2> 3402708 INFO (jetty-launcher-6536-thread-1) [n:127.0.0.1:63485_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63485_solr
[junit4] 2> 3402710 INFO (zkCallback-6552-thread-1-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3402711 INFO (zkCallback-6554-thread-1-processing-n:127.0.0.1:63485_solr) [n:127.0.0.1:63485_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3402720 INFO (jetty-launcher-6536-thread-2) [n:127.0.0.1:51759_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51759_solr
[junit4] 2> 3402722 INFO (zkCallback-6552-thread-1-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3402725 INFO (zkCallback-6554-thread-1-processing-n:127.0.0.1:63485_solr) [n:127.0.0.1:63485_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3402854 INFO (jetty-launcher-6536-thread-2) [n:127.0.0.1:51759_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3402859 INFO (jetty-launcher-6536-thread-2) [n:127.0.0.1:51759_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3402860 INFO (jetty-launcher-6536-thread-2) [n:127.0.0.1:51759_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3402860 INFO (jetty-launcher-6536-thread-2) [n:127.0.0.1:51759_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node1/.
[junit4] 2> 3402873 INFO (jetty-launcher-6536-thread-1) [n:127.0.0.1:63485_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63485.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3402878 INFO (jetty-launcher-6536-thread-1) [n:127.0.0.1:63485_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63485.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3402878 INFO (jetty-launcher-6536-thread-1) [n:127.0.0.1:63485_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63485.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3402879 INFO (jetty-launcher-6536-thread-1) [n:127.0.0.1:63485_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node2/.
[junit4] 2> 3402895 INFO (zkConnectionManagerCallback-6561-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3402899 INFO (zkConnectionManagerCallback-6565-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3402901 INFO (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3402904 INFO (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54360/solr ready
[junit4] 2> 3402910 INFO (SUITE-TestTlogReplica-seed#[B34AFF72C3CCB2C9]-worker) [ ] o.a.s.c.TestTlogReplica Using legacyCloud?: false
[junit4] 2> 3402911 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3402912 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3402928 INFO (TEST-TestTlogReplica.testCreateDelete-seed#[B34AFF72C3CCB2C9]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateDelete {seed=[B34AFF72C3CCB2C9:A85A0E3E353C4AF9]}
[junit4] 2> 3402934 INFO (OverseerThreadFactory-12042-thread-1-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection tlog_replica_test_create_delete
[junit4] 2> 3402935 WARN (OverseerThreadFactory-12042-thread-1-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 4 on collection tlog_replica_test_create_delete is higher than the number of Solr instances currently live or live and part of your createNodeSet(2). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 3403048 INFO (OverseerStateUpdate-72229147587379205-127.0.0.1:51759_solr-n_0000000000) [n:127.0.0.1:51759_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_create_delete",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_create_delete_shard1_replica_t1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:63485/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3403052 INFO (OverseerStateUpdate-72229147587379205-127.0.0.1:51759_solr-n_0000000000) [n:127.0.0.1:51759_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_create_delete",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_create_delete_shard1_replica_t2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51759/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3403055 INFO (OverseerStateUpdate-72229147587379205-127.0.0.1:51759_solr-n_0000000000) [n:127.0.0.1:51759_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_create_delete",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_create_delete_shard1_replica_t4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:63485/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3403058 INFO (OverseerStateUpdate-72229147587379205-127.0.0.1:51759_solr-n_0000000000) [n:127.0.0.1:51759_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_create_delete",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_create_delete_shard1_replica_t6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51759/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3403062 INFO (OverseerStateUpdate-72229147587379205-127.0.0.1:51759_solr-n_0000000000) [n:127.0.0.1:51759_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_create_delete",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"tlog_replica_test_create_delete_shard2_replica_t8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:63485/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3403066 INFO (OverseerStateUpdate-72229147587379205-127.0.0.1:51759_solr-n_0000000000) [n:127.0.0.1:51759_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_create_delete",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"tlog_replica_test_create_delete_shard2_replica_t10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51759/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3403069 INFO (OverseerStateUpdate-72229147587379205-127.0.0.1:51759_solr-n_0000000000) [n:127.0.0.1:51759_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_create_delete",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"tlog_replica_test_create_delete_shard2_replica_t12",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:63485/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3403073 INFO (OverseerStateUpdate-72229147587379205-127.0.0.1:51759_solr-n_0000000000) [n:127.0.0.1:51759_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_create_delete",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"tlog_replica_test_create_delete_shard2_replica_t14",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51759/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3403277 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard1_replica_t1&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 3403278 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3403279 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard1_replica_t2&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 3403279 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3403279 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard1_replica_t4&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 3403280 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard1_replica_t6&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 3403280 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node13&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard2_replica_t10&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 3403281 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node15&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard2_replica_t12&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 3403281 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard2_replica_t8&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 3403281 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node16&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard2_replica_t14&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 3403415 INFO (zkCallback-6554-thread-1-processing-n:127.0.0.1:63485_solr) [n:127.0.0.1:63485_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3403415 INFO (zkCallback-6554-thread-2-processing-n:127.0.0.1:63485_solr) [n:127.0.0.1:63485_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3403415 INFO (zkCallback-6552-thread-2-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3403415 INFO (zkCallback-6552-thread-1-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3404305 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3404309 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3404316 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3404317 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3404335 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3404335 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3404335 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3404343 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3404344 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.s.IndexSchema [tlog_replica_test_create_delete_shard1_replica_t6] Schema name=minimal
[junit4] 2> 3404345 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.s.IndexSchema [tlog_replica_test_create_delete_shard1_replica_t1] Schema name=minimal
[junit4] 2> 3404346 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3404347 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_create_delete_shard1_replica_t6' using configuration from collection tlog_replica_test_create_delete, trusted=true
[junit4] 2> 3404347 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.core.tlog_replica_test_create_delete.shard1.replica_t6' (registry 'solr.core.tlog_replica_test_create_delete.shard1.replica_t6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3404347 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3404347 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.c.SolrCore [[tlog_replica_test_create_delete_shard1_replica_t6] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node1/tlog_replica_test_create_delete_shard1_replica_t6], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node1/./tlog_replica_test_create_delete_shard1_replica_t6/data/]
[junit4] 2> 3404349 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3404349 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_create_delete_shard1_replica_t1' using configuration from collection tlog_replica_test_create_delete, trusted=true
[junit4] 2> 3404349 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63485.solr.core.tlog_replica_test_create_delete.shard1.replica_t1' (registry 'solr.core.tlog_replica_test_create_delete.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3404349 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3404350 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.c.SolrCore [[tlog_replica_test_create_delete_shard1_replica_t1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node2/tlog_replica_test_create_delete_shard1_replica_t1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node2/./tlog_replica_test_create_delete_shard1_replica_t1/data/]
[junit4] 2> 3404350 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.s.IndexSchema [tlog_replica_test_create_delete_shard2_replica_t8] Schema name=minimal
[junit4] 2> 3404352 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3404352 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_create_delete_shard2_replica_t8' using configuration from collection tlog_replica_test_create_delete, trusted=true
[junit4] 2> 3404352 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63485.solr.core.tlog_replica_test_create_delete.shard2.replica_t8' (registry 'solr.core.tlog_replica_test_create_delete.shard2.replica_t8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3404352 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3404352 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.SolrCore [[tlog_replica_test_create_delete_shard2_replica_t8] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node2/tlog_replica_test_create_delete_shard2_replica_t8], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node2/./tlog_replica_test_create_delete_shard2_replica_t8/data/]
[junit4] 2> 3404360 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.s.IndexSchema [tlog_replica_test_create_delete_shard2_replica_t10] Schema name=minimal
[junit4] 2> 3404373 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.s.IndexSchema [tlog_replica_test_create_delete_shard1_replica_t2] Schema name=minimal
[junit4] 2> 3404373 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3404374 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_create_delete_shard2_replica_t10' using configuration from collection tlog_replica_test_create_delete, trusted=true
[junit4] 2> 3404374 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.core.tlog_replica_test_create_delete.shard2.replica_t10' (registry 'solr.core.tlog_replica_test_create_delete.shard2.replica_t10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3404374 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3404374 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.c.SolrCore [[tlog_replica_test_create_delete_shard2_replica_t10] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node1/tlog_replica_test_create_delete_shard2_replica_t10], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node1/./tlog_replica_test_create_delete_shard2_replica_t10/data/]
[junit4] 2> 3404380 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.s.IndexSchema [tlog_replica_test_create_delete_shard2_replica_t14] Schema name=minimal
[junit4] 2> 3404382 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.s.IndexSchema [tlog_replica_test_create_delete_shard1_replica_t4] Schema name=minimal
[junit4] 2> 3404384 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3404384 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_create_delete_shard1_replica_t4' using configuration from collection tlog_replica_test_create_delete, trusted=true
[junit4] 2> 3404385 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63485.solr.core.tlog_replica_test_create_delete.shard1.replica_t4' (registry 'solr.core.tlog_replica_test_create_delete.shard1.replica_t4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3404385 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3404385 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.SolrCore [[tlog_replica_test_create_delete_shard1_replica_t4] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node2/tlog_replica_test_create_delete_shard1_replica_t4], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node2/./tlog_replica_test_create_delete_shard1_replica_t4/data/]
[junit4] 2> 3404390 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3404390 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_create_delete_shard1_replica_t2' using configuration from collection tlog_replica_test_create_delete, trusted=true
[junit4] 2> 3404390 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.core.tlog_replica_test_create_delete.shard1.replica_t2' (registry 'solr.core.tlog_replica_test_create_delete.shard1.replica_t2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3404390 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3404390 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.c.SolrCore [[tlog_replica_test_create_delete_shard1_replica_t2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node1/tlog_replica_test_create_delete_shard1_replica_t2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node1/./tlog_replica_test_create_delete_shard1_replica_t2/data/]
[junit4] 2> 3404402 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.s.IndexSchema [tlog_replica_test_create_delete_shard2_replica_t12] Schema name=minimal
[junit4] 2> 3404410 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3404410 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_create_delete_shard2_replica_t12' using configuration from collection tlog_replica_test_create_delete, trusted=true
[junit4] 2> 3404411 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63485.solr.core.tlog_replica_test_create_delete.shard2.replica_t12' (registry 'solr.core.tlog_replica_test_create_delete.shard2.replica_t12') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3404411 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3404411 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.c.SolrCore [[tlog_replica_test_create_delete_shard2_replica_t12] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node2/tlog_replica_test_create_delete_shard2_replica_t12], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node2/./tlog_replica_test_create_delete_shard2_replica_t12/data/]
[junit4] 2> 3404427 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3404427 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_create_delete_shard2_replica_t14' using configuration from collection tlog_replica_test_create_delete, trusted=true
[junit4] 2> 3404427 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.core.tlog_replica_test_create_delete.shard2.replica_t14' (registry 'solr.core.tlog_replica_test_create_delete.shard2.replica_t14') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79081443
[junit4] 2> 3404427 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3404427 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.c.SolrCore [[tlog_replica_test_create_delete_shard2_replica_t14] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node1/tlog_replica_test_create_delete_shard2_replica_t14], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001/tempDir-001/node1/./tlog_replica_test_create_delete_shard2_replica_t14/data/]
[junit4] 2> 3404475 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3404475 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3404479 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3404479 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3404481 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.s.SolrIndexSearcher Opening [Searcher@4c405217[tlog_replica_test_create_delete_shard2_replica_t8] main]
[junit4] 2> 3404484 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3404485 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3404485 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3404486 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592892284951592960
[junit4] 2> 3404500 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=4 found=1 timeoutin=9999ms
[junit4] 2> 3404509 INFO (searcherExecutor-12049-thread-1-processing-n:127.0.0.1:63485_solr x:tlog_replica_test_create_delete_shard2_replica_t8 s:shard2 c:tlog_replica_test_create_delete r:core_node11) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.SolrCore [tlog_replica_test_create_delete_shard2_replica_t8] Registered new searcher Searcher@4c405217[tlog_replica_test_create_delete_shard2_replica_t8] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3404529 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3404530 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3404536 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3404536 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3404552 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.s.SolrIndexSearcher Opening [Searcher@72adf39b[tlog_replica_test_create_delete_shard1_replica_t4] main]
[junit4] 2> 3404553 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3404553 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3404556 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3404556 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592892285024993280
[junit4] 2> 3404562 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3404563 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3404564 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3404564 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3404566 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.s.SolrIndexSearcher Opening [Searcher@13a42f93[tlog_replica_test_create_delete_shard1_replica_t6] main]
[junit4] 2> 3404571 INFO (searcherExecutor-12051-thread-1-processing-n:127.0.0.1:63485_solr x:tlog_replica_test_create_delete_shard1_replica_t4 s:shard1 c:tlog_replica_test_create_delete r:core_node7) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.SolrCore [tlog_replica_test_create_delete_shard1_replica_t4] Registered new searcher Searcher@72adf39b[tlog_replica_test_create_delete_shard1_replica_t4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3404574 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3404574 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3404576 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3404576 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3404577 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.s.SolrIndexSearcher Opening [Searcher@4bef01ee[tlog_replica_test_create_delete_shard2_replica_t14] main]
[junit4] 2> 3404578 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=4 found=1 timeoutin=9999ms
[junit4] 2> 3404578 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3404578 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3404578 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3404578 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3404580 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3404580 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3404580 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3404580 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3404581 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.s.SolrIndexSearcher Opening [Searcher@7a987d7d[tlog_replica_test_create_delete_shard2_replica_t10] main]
[junit4] 2> 3404582 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3404582 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3404581 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@4aad596f[tlog_replica_test_create_delete_shard1_replica_t2] main]
[junit4] 2> 3404582 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3404582 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3404583 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3404583 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3404583 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592892285053304832
[junit4] 2> 3404583 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3404583 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3404583 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592892285053304832
[junit4] 2> 3404584 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3404584 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3404584 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592892285054353408
[junit4] 2> 3404585 INFO (searcherExecutor-12047-thread-1-processing-n:127.0.0.1:51759_solr x:tlog_replica_test_create_delete_shard1_replica_t6 s:shard1 c:tlog_replica_test_create_delete r:core_node9) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.c.SolrCore [tlog_replica_test_create_delete_shard1_replica_t6] Registered new searcher Searcher@13a42f93[tlog_replica_test_create_delete_shard1_replica_t6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3404585 INFO (searcherExecutor-12054-thread-1-processing-n:127.0.0.1:51759_solr x:tlog_replica_test_create_delete_shard2_replica_t14 s:shard2 c:tlog_replica_test_create_delete r:core_node16) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.c.SolrCore [tlog_replica_test_create_delete_shard2_replica_t14] Registered new searcher Searcher@4bef01ee[tlog_replica_test_create_delete_shard2_replica_t14] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3404585 INFO (searcherExecutor-12052-thread-1-processing-n:127.0.0.1:51759_solr x:tlog_replica_test_create_delete_shard1_replica_t2 s:shard1 c:tlog_replica_test_create_delete r:core_node5) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_create_delete_shard1_replica_t2] Registered new searcher Searcher@4aad596f[tlog_replica_test_create_delete_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3404586 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3404586 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3404586 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3404586 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3404587 INFO (searcherExecutor-12050-thread-1-processing-n:127.0.0.1:51759_solr x:tlog_replica_test_create_delete_shard2_replica_t10 s:shard2 c:tlog_replica_test_create_delete r:core_node13) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.c.SolrCore [tlog_replica_test_create_delete_shard2_replica_t10] Registered new searcher Searcher@7a987d7d[tlog_replica_test_create_delete_shard2_replica_t10] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3404587 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592892285057499136
[junit4] 2> 3404588 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3404588 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3404589 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.s.SolrIndexSearcher Opening [Searcher@3a99befc[tlog_replica_test_create_delete_shard2_replica_t12] main]
[junit4] 2> 3404590 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3404591 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3404591 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3404591 INFO (qtp1671251517-33166) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.c.ZkShardTerms Failed to save terms, version is not match, retrying
[junit4] 2> 3404592 INFO (searcherExecutor-12053-thread-1-processing-n:127.0.0.1:63485_solr x:tlog_replica_test_create_delete_shard2_replica_t12 s:shard2 c:tlog_replica_test_create_delete r:core_node15) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.c.SolrCore [tlog_replica_test_create_delete_shard2_replica_t12] Registered new searcher Searcher@3a99befc[tlog_replica_test_create_delete_shard2_replica_t12] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3404592 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592892285062742016
[junit4] 2> 3404594 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.c.ZkShardTerms Failed to save terms, version is not match, retrying
[junit4] 2> 3404594 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.c.ZkShardTerms Failed to save terms, version is not match, retrying
[junit4] 2> 3404601 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3404601 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3404603 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3404603 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3404604 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@482c1713[tlog_replica_test_create_delete_shard1_replica_t1] main]
[junit4] 2> 3404606 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3404606 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3404607 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3404607 INFO (searcherExecutor-12048-thread-1-processing-n:127.0.0.1:63485_solr x:tlog_replica_test_create_delete_shard1_replica_t1 s:shard1 c:tlog_replica_test_create_delete r:core_node3) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_create_delete_shard1_replica_t1] Registered new searcher Searcher@482c1713[tlog_replica_test_create_delete_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3404607 INFO (qtp100832216-33163) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592892285078470656
[junit4] 2> 3404683 INFO (zkCallback-6554-thread-2-processing-n:127.0.0.1:63485_solr) [n:127.0.0.1:63485_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3404683 INFO (zkCallback-6552-thread-1-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3404683 INFO (zkCallback-6554-thread-1-processing-n:127.0.0.1:63485_solr) [n:127.0.0.1:63485_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3404683 INFO (zkCallback-6552-thread-2-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3405001 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3405001 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3405001 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63485/solr/tlog_replica_test_create_delete_shard2_replica_t8/
[junit4] 2> 3405002 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_create_delete_shard2_replica_t8 url=http://127.0.0.1:63485/solr START replicas=[http://127.0.0.1:51759/solr/tlog_replica_test_create_delete_shard2_replica_t10/, http://127.0.0.1:63485/solr/tlog_replica_test_create_delete_shard2_replica_t12/, http://127.0.0.1:51759/solr/tlog_replica_test_create_delete_shard2_replica_t14/] nUpdates=100
[junit4] 2> 3405005 INFO (qtp1671251517-33172) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.c.S.Request [tlog_replica_test_create_delete_shard2_replica_t10] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3405006 INFO (qtp100832216-33165) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.c.S.Request [tlog_replica_test_create_delete_shard2_replica_t12] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3405007 INFO (qtp1671251517-33173) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.c.S.Request [tlog_replica_test_create_delete_shard2_replica_t14] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3405079 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3405080 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3405080 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63485/solr/tlog_replica_test_create_delete_shard1_replica_t4/
[junit4] 2> 3405080 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_create_delete_shard1_replica_t4 url=http://127.0.0.1:63485/solr START replicas=[http://127.0.0.1:63485/solr/tlog_replica_test_create_delete_shard1_replica_t1/, http://127.0.0.1:51759/solr/tlog_replica_test_create_delete_shard1_replica_t2/, http://127.0.0.1:51759/solr/tlog_replica_test_create_delete_shard1_replica_t6/] nUpdates=100
[junit4] 2> 3405110 INFO (qtp100832216-33161) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node3 x:tlog_replica_test_create_delete_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_create_delete_shard1_replica_t1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3405110 INFO (qtp1671251517-33170) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node5 x:tlog_replica_test_create_delete_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_create_delete_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3405111 INFO (qtp1671251517-33263) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.c.S.Request [tlog_replica_test_create_delete_shard1_replica_t6] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 3405308 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_create_delete_shard2_replica_t8 url=http://127.0.0.1:63485/solr DONE. We have no versions. sync failed.
[junit4] 2> 3405308 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 3405308 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 3405308 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3405309 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.ZkController tlog_replica_test_create_delete_shard2_replica_t8 stopping background replication from leader
[junit4] 2> 3405313 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63485/solr/tlog_replica_test_create_delete_shard2_replica_t8/ shard2
[junit4] 2> 3405381 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_create_delete_shard1_replica_t4 url=http://127.0.0.1:63485/solr DONE. We have no versions. sync failed.
[junit4] 2> 3405381 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 3405382 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 3405382 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3405382 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.ZkController tlog_replica_test_create_delete_shard1_replica_t4 stopping background replication from leader
[junit4] 2> 3405387 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63485/solr/tlog_replica_test_create_delete_shard1_replica_t4/ shard1
[junit4] 2> 3405489 INFO (zkCallback-6552-thread-2-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3405489 INFO (zkCallback-6554-thread-1-processing-n:127.0.0.1:63485_solr) [n:127.0.0.1:63485_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3405489 INFO (zkCallback-6552-thread-1-processing-n:127.0.0.1:51759_solr) [n:127.0.0.1:51759_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3405489 INFO (zkCallback-6554-thread-2-processing-n:127.0.0.1:63485_solr) [n:127.0.0.1:63485_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_create_delete/state.json] for collection [tlog_replica_test_create_delete] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3405515 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3405517 INFO (qtp100832216-33158) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node11 x:tlog_replica_test_create_delete_shard2_replica_t8] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard2_replica_t8&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2236
[junit4] 2> 3405538 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3405541 INFO (qtp100832216-33164) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard1 r:core_node7 x:tlog_replica_test_create_delete_shard1_replica_t4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=tlog_replica_test_create_delete_shard1_replica_t4&action=CREATE&numShards=2&collection=tlog_replica_test_create_delete&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2262
[junit4] 2> 3405600 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.c.ZkController tlog_replica_test_create_delete_shard2_replica_t12 starting background replication from leader
[junit4] 2> 3405600 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 3405600 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.c.ZkController tlog_replica_test_create_delete_shard1_replica_t6 starting background replication from leader
[junit4] 2> 3405600 INFO (qtp1671251517-33171) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard1 r:core_node9 x:tlog_replica_test_create_delete_shard1_replica_t6] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 3405601 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.c.ZkController tlog_replica_test_create_delete_shard2_replica_t14 starting background replication from leader
[junit4] 2> 3405601 INFO (qtp1671251517-33174) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node16 x:tlog_replica_test_create_delete_shard2_replica_t14] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 3405602 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 3405602 INFO (qtp100832216-33160) [n:127.0.0.1:63485_solr c:tlog_replica_test_create_delete s:shard2 r:core_node15 x:tlog_replica_test_create_delete_shard2_replica_t12] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3405602 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.c.ZkController tlog_replica_test_create_delete_shard2_replica_t10 starting background replication from leader
[junit4] 2> 3405602 INFO (qtp1671251517-33169) [n:127.0.0.1:51759_solr c:tlog_replica_test_create_delete s:shard2 r:core_node13 x:tlog_replica_test_create_delete_shard2_replica_t10] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 3405602 INFO (qtp1
[...truncated too long message...]
2> 15 /solr/collections/tlog_replica_test_only_leader_indexes/state.json
[junit4] 2> 13 /solr/collections/tlog_replica_test_add_docs/state.json
[junit4] 2> 13 /solr/collections/tlog_replica_test_real_time_get/state.json
[junit4] 2> 13 /solr/collections/tlog_replica_test_delete_by_id/state.json
[junit4] 2> 10 /solr/clusterstate.json
[junit4] 2> 10 /solr/clusterprops.json
[junit4] 2> 4 /solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/72229147587379217-core_node3-n_0000000000
[junit4] 2> 4 /solr/autoscaling.json
[junit4] 2> 3 /solr/overseer_elect/election/72229147587379206-127.0.0.1:63485_solr-n_0000000001
[junit4] 2> 3 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72229147587379206-core_node11-n_0000000000
[junit4] 2> 3 /solr/overseer_elect/election/72229147587379217-127.0.0.1:51759_solr-n_0000000004
[junit4] 2> 2 /solr/collections/tlog_replica_test_kill_tlog_replica/leader_elect/shard1/election/72229147587379206-core_node4-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72229147587379206-core_node13-n_0000000001
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72229147587379206-core_node5-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72229147587379206-core_node7-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/collections
[junit4] 2> 9 /solr/live_nodes
[junit4] 2> 4 /solr/overseer/queue
[junit4] 2> 4 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2> 4 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_B34AFF72C3CCB2C9-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {foo=Lucene50(blocksize=128), title_s=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), foo_s=Lucene50(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Direct), inplace_updatable_int=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1236, maxMBSortInHeap=7.1236066006673555, sim=RandomSimilarity(queryNorm=true): {}, locale=ms-MY, timezone=America/Guatemala
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=103516952,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [HdfsNNFailoverTest, PeerSyncTest, DistributedQueueTest, LeaderInitiatedRecoveryOnShardRestartTest, TestCoreDiscovery, TestPHPSerializedResponseWriter, ResponseHeaderTest, ClusterStateUpdateTest, TestExportWriter, TestBadConfig, CdcrUpdateLogTest, SpellCheckComponentTest, DistributedFacetSimpleRefinementLongTailTest, HdfsTlogReplayBufferedWhileIndexingTest, TestXmlQParser, CircularListTest, SimpleFacetsTest, TestReloadAndDeleteDocs, TestPKIAuthenticationPlugin, UUIDFieldTest, TestLuceneIndexBackCompat, TestConfig, TestManagedSchemaThreadSafety, PeerSyncReplicationTest, ResponseLogComponentTest, ChaosMonkeySafeLeaderWithPullReplicasTest, TestDownShardTolerantSearch, URLClassifyProcessorTest, PreAnalyzedFieldTest, DistanceUnitsTest, TestUpdate, ZkStateWriterTest, TestInitParams, AliasIntegrationTest, TestSolrIndexConfig, TestPushWriter, MetricTriggerTest, DOMUtilTest, TestStressInPlaceUpdates, OverriddenZkACLAndCredentialsProvidersTest, TestDistribIDF, TestManagedResourceStorage, BlobRepositoryCloudTest, ConcurrentDeleteAndCreateCollectionTest, CoreAdminRequestStatusTest, CachingDirectoryFactoryTest, TestFieldCacheVsDocValues, TestMaxTokenLenTokenizer, CheckHdfsIndexTest, JavabinLoaderTest, JSONWriterTest, SoftAutoCommitTest, QueryParsingTest, TestFuzzyAnalyzedSuggestions, TestLegacyTerms, TestJsonFacets, TestNodeLostTrigger, DistributedFacetPivotLargeTest, BasicAuthIntegrationTest, SyncSliceTest, TestStressLucene, ChaosMonkeySafeLeaderTest, TestRTimerTree, SpatialHeatmapFacetsTest, CoreSorterTest, AutoAddReplicasPlanActionTest, TolerantUpdateProcessorTest, TestMultiWordSynonyms, TestBulkSchemaAPI, MinimalSchemaTest, SolrJmxReporterCloudTest, TestApiFramework, ManagedSchemaRoundRobinCloudTest, TestIndexingPerformance, TestCloudManagedSchema, InfoHandlerTest, BooleanFieldTest, TestDocSet, SystemInfoHandlerTest, ConcurrentCreateRoutedAliasTest, TestHdfsUpdateLog, SolrShardReporterTest, SegmentsInfoRequestHandlerTest, TestInitQParser, CursorMarkTest, SuggesterWFSTTest, TestWriterPerf, TestIntervalFaceting, TestDocumentBuilder, PeerSyncWithIndexFingerprintCachingTest, TestExecutePlanAction, DocExpirationUpdateProcessorFactoryTest, UninvertDocValuesMergePolicyTest, TestFieldTypeCollectionResource, TestRandomDVFaceting, TestDFRSimilarityFactory, DocValuesMultiTest, DocValuesMissingTest, SolrMetricReporterTest, TestMaxScoreQueryParser, AddBlockUpdateTest, PrimitiveFieldTypeTest, TestImplicitCoreProperties, TestCloudInspectUtil, TestClusterProperties, TlogReplayBufferedWhileIndexingTest, DirectUpdateHandlerTest, TestNamedUpdateProcessors, DocValuesNotIndexedTest, TestReplicationHandlerBackup, SpellPossibilityIteratorTest, TestFieldCacheSort, TestSolrFieldCacheBean, TestLuceneMatchVersion, DebugComponentTest, TestSolrCoreProperties, ChaosMonkeyNothingIsSafeTest, TestComplexPhraseQParserPlugin, TestSubQueryTransformer, AutoscalingHistoryHandlerTest, CloudMLTQParserTest, ScriptEngineTest, TestJettySolrRunner, TestDistributedSearch, TestSchemaManager, TestStressRecovery, TestSizeLimitedDistributedMap, OverseerStatusTest, CurrencyFieldTypeTest, VersionInfoTest, DistributedIntervalFacetingTest, TestDynamicFieldCollectionResource, CollectionStateFormat2Test, RemoteQueryErrorTest, TestRequestForwarding, UpdateRequestProcessorFactoryTest, ExplicitHLLTest, TestLargeCluster, JvmMetricsTest, VMParamsZkACLAndCredentialsProvidersTest, ZkControllerTest, TestSearchPerf, TestSegmentSorting, TestAnalyzedSuggestions, DeleteStatusTest, TestConfigSetProperties, TestLegacyFieldReuse, FieldAnalysisRequestHandlerTest, TestNonDefinedSimilarityFactory, TestReRankQParserPlugin, NoCacheHeaderTest, HdfsWriteToMultipleCollectionsTest, TestStressUserVersions, TestConfigOverlay, SimpleCollectionCreateDeleteTest, TestFastWriter, MoveReplicaHDFSFailoverTest, TestPrepRecovery, TestPayloadScoreQParserPlugin, ClassificationUpdateProcessorFactoryTest, TestSolr4Spatial2, SolrSlf4jReporterTest, TestZkAclsWithHadoopAuth, TestCollectionsAPIViaSolrCloudCluster, QueryResultKeyTest, AtomicUpdateProcessorFactoryTest, ParsingFieldUpdateProcessorsTest, BadIndexSchemaTest, TestExactStatsCache, TestReloadDeadlock, TestManagedSchema, TestDistributedGrouping, TestRTGBase, TestUnifiedSolrHighlighter, TestSolrCoreSnapshots, TestConfigSetsAPIExclusivity, SchemaApiFailureTest, SolrCoreMetricManagerTest, TestStressVersions, TestNestedDocsSort, TestClassicSimilarityFactory, ZkShardTermsTest, TestCoreBackup, ConnectionManagerTest, ActionThrottleTest, TestComputePlanAction, TestComponentsName, HdfsChaosMonkeyNothingIsSafeTest, TestSort, LoggingHandlerTest, ExitableDirectoryReaderTest, TestAuthenticationFramework, TestCSVLoader, BasicDistributedZk2Test, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, ZkCLITest, ShardRoutingCustomTest, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestRealTimeGet, TestStressReorder, TestJoin, DistributedTermsComponentTest, TestSolr4Spatial, SolrCoreTest, SolrCmdDistributorTest, TestFunctionQuery, BasicFunctionalityTest, ShowFileRequestHandlerTest, SolrIndexSplitterTest, SimplePostToolTest, TestExtendedDismaxParser, SpellCheckCollatorTest, CoreAdminHandlerTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestWordDelimiterFilterFactory, StatelessScriptUpdateProcessorFactoryTest, XsltUpdateRequestHandlerTest, DisMaxRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TermsComponentTest, RequiredFieldsTest, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, TestJmxIntegration, MBeansHandlerTest, BadComponentTest, TestSolrDeletionPolicy2, MultiTermTest, NumericFieldsTest, OutputWriterTest, NotRequiredUniqueKeyTest, TestCharFilters, TestCodecSupport, TestLMJelinekMercerSimilarityFactory, TestFastOutputStream, TestFastLRUCache, PrimUtilsTest, RAMDirectoryFactoryTest, ClusterStateTest, TestUtils, SliceStateTest, FileUtilsTest, DistributedMLTComponentTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, AssignBackwardCompatibilityTest, ConfigSetsAPITest, CreateCollectionCleanupTest, CreateRoutedAliasTest, DeleteLastCustomShardedReplicaTest, DeleteReplicaTest, DistribDocExpirationUpdateProcessorTest, LIRRollingUpdatesTest, MoveReplicaHDFSTest, NodeMutatorTest, OutOfBoxZkACLAndCredentialsProvidersTest, RecoveryAfterSoftCommitTest, ReplaceNodeNoTargetTest, ReplaceNodeTest, SSLMigrationTest, SharedFSAutoReplicaFailoverTest, SolrCLIZkUtilsTest, SolrCloudExampleTest, TestCloudPivotFacet, TestCloudSearcherWarming, TestCryptoKeys, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestMiniSolrCloudClusterSSL, TestPullReplicaErrorHandling, TestShortCircuitedRequests, TestSolrCloudWithDelegationTokens, TestSolrCloudWithSecureImpersonation, TestStressCloudBlindAtomicUpdates, TestTlogReplica]
[junit4] Completed [581/777 (1!)] on J1 in 122.73s, 13 tests, 1 failure <<< FAILURES!
[...truncated 49191 lines...]
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 457 -
Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/457/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=30214, name=jetty-launcher-5224-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) 2) Thread[id=30218, name=jetty-launcher-5224-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=30214, name=jetty-launcher-5224-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
2) Thread[id=30218, name=jetty-launcher-5224-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
at __randomizedtesting.SeedInfo.seed([DC61E2C91E3D03BF]:0)
Build Log:
[...truncated 13913 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_DC61E2C91E3D03BF-001/init-core-data-001
[junit4] 2> 3447467 WARN (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 3447467 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3447468 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 3447470 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 3447494 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_DC61E2C91E3D03BF-001/tempDir-001
[junit4] 2> 3447494 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3447494 INFO (Thread-10284) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3447494 INFO (Thread-10284) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3447498 ERROR (Thread-10284) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3447594 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:44351
[junit4] 2> 3447601 INFO (zkConnectionManagerCallback-5227-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3447615 INFO (jetty-launcher-5224-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3447615 INFO (jetty-launcher-5224-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3447616 INFO (jetty-launcher-5224-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3447616 INFO (jetty-launcher-5224-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3447616 INFO (jetty-launcher-5224-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 3447616 INFO (jetty-launcher-5224-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2773f3d4{/solr,null,AVAILABLE}
[junit4] 2> 3447617 INFO (jetty-launcher-5224-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3447617 INFO (jetty-launcher-5224-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3447617 INFO (jetty-launcher-5224-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 3447617 INFO (jetty-launcher-5224-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@58ceaae{/solr,null,AVAILABLE}
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@54d8dbc8{SSL,[ssl, http/1.1]}{127.0.0.1:63222}
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7d2ca322{SSL,[ssl, http/1.1]}{127.0.0.1:40144}
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-1) [ ] o.e.j.s.Server Started @3455342ms
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40144}
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-2) [ ] o.e.j.s.Server Started @3455342ms
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63222}
[junit4] 2> 3447618 ERROR (jetty-launcher-5224-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3447618 ERROR (jetty-launcher-5224-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T18:52:28.802Z
[junit4] 2> 3447618 INFO (jetty-launcher-5224-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T18:52:28.802Z
[junit4] 2> 3447621 INFO (zkConnectionManagerCallback-5231-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3447621 INFO (zkConnectionManagerCallback-5229-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3447621 INFO (jetty-launcher-5224-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3447621 INFO (jetty-launcher-5224-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3447629 INFO (jetty-launcher-5224-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44351/solr
[junit4] 2> 3447629 INFO (jetty-launcher-5224-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44351/solr
[junit4] 2> 3447630 INFO (zkConnectionManagerCallback-5237-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3447630 INFO (zkConnectionManagerCallback-5239-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3447633 INFO (zkConnectionManagerCallback-5241-thread-1-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3447633 INFO (zkConnectionManagerCallback-5243-thread-1-processing-n:127.0.0.1:40144_solr) [n:127.0.0.1:40144_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3447765 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3447769 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:63222_solr
[junit4] 2> 3447769 INFO (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3447770 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.c.Overseer Overseer (id=72243918190280709-127.0.0.1:63222_solr-n_0000000000) starting
[junit4] 2> 3447771 INFO (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40144_solr
[junit4] 2> 3447774 INFO (zkCallback-5242-thread-1-processing-n:127.0.0.1:40144_solr) [n:127.0.0.1:40144_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3447777 INFO (zkCallback-5240-thread-1-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3447787 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63222_solr
[junit4] 2> 3447788 INFO (zkCallback-5242-thread-1-processing-n:127.0.0.1:40144_solr) [n:127.0.0.1:40144_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3447789 INFO (zkCallback-5240-thread-1-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3447821 INFO (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 3447822 WARN (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3447824 INFO (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 3447829 INFO (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 3447845 INFO (jetty-launcher-5224-thread-1-EventThread) [n:127.0.0.1:40144_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 3447900 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 3447900 WARN (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3447900 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 3447902 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 3447904 INFO (jetty-launcher-5224-thread-2-EventThread) [n:127.0.0.1:63222_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 3447933 WARN (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3447966 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63222.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d5fb93
[junit4] 2> 3447971 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63222.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d5fb93
[junit4] 2> 3447971 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63222.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d5fb93
[junit4] 2> 3447972 INFO (jetty-launcher-5224-thread-2) [n:127.0.0.1:63222_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_DC61E2C91E3D03BF-001/tempDir-001/node2/.
[junit4] 2> 3448867 WARN (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3448899 INFO (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40144.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d5fb93
[junit4] 2> 3448905 INFO (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40144.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d5fb93
[junit4] 2> 3448905 INFO (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40144.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d5fb93
[junit4] 2> 3448906 INFO (jetty-launcher-5224-thread-1) [n:127.0.0.1:40144_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_DC61E2C91E3D03BF-001/tempDir-001/node1/.
[junit4] 2> 3448927 INFO (zkConnectionManagerCallback-5249-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3448929 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a975f8ea0009, likely client has closed socket
[junit4] 2> 3449080 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
[junit4] 2> 3459150 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
[junit4] 2> 3459300 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
[junit4] 2> 3459321 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
[junit4] 2> 3459478 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
[junit4] 2> 3459479 INFO (qtp1265219449-30169) [n:127.0.0.1:63222_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3459479 INFO (qtp1265219449-30169) [n:127.0.0.1:63222_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3459480 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
[junit4] 2> 3459628 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
[junit4] 2> 3459629 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3459629 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3459629 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
[junit4] 2> 3459787 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
[junit4] 2> 3459788 INFO (qtp1265219449-30169) [n:127.0.0.1:63222_solr ] o.a.s.s.SolrDispatchFilter Error authenticating
[junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:454)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:289)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:149)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3459788 WARN (qtp1265219449-30169) [n:127.0.0.1:63222_solr ] o.e.j.s.HttpChannel /solr/admin/collections
[junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:460)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:289)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:149)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:454)
[junit4] 2> ... 34 more
[junit4] 2> 3459790 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
[junit4] 2> 3459970 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
[junit4] 2> 3460002 WARN (qtp1265219449-30169) [n:127.0.0.1:63222_solr ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
[junit4] 2> id: invalid user name: "bar"
[junit4] 2>
[junit4] 2> 3460004 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
[junit4] 2> 3460172 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
[junit4] 2> 3460195 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
[junit4] 2> 3460397 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
[junit4] 2> 3460399 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
[junit4] 2> 3460578 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
[junit4] 2> 3476102 WARN (jetty-launcher-5224-thread-2-SendThread(127.0.0.1:44351)) [n:127.0.0.1:63222_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17164ms for sessionid 0x100a975f8ea0008
[junit4] 2> 3476102 WARN (jetty-launcher-5224-thread-1-SendThread(127.0.0.1:44351)) [n:127.0.0.1:40144_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17164ms for sessionid 0x100a975f8ea0007
[junit4] 2> 3476102 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a975f8ea0008, likely client has closed socket
[junit4] 2> 3476102 WARN (jetty-launcher-5224-thread-1-SendThread(127.0.0.1:44351)) [n:127.0.0.1:40144_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17164ms for sessionid 0x100a975f8ea0006
[junit4] 2> 3476102 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a975f8ea0007, likely client has closed socket
[junit4] 2> 3476102 WARN (jetty-launcher-5224-thread-2-SendThread(127.0.0.1:44351)) [n:127.0.0.1:63222_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17164ms for sessionid 0x100a975f8ea0005
[junit4] 2> 3476102 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a975f8ea0006, likely client has closed socket
[junit4] 2> 3476103 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a975f8ea0005, likely client has closed socket
[junit4] 2> 3476103 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
[junit4] 2> 3476236 WARN (zkConnectionManagerCallback-5243-thread-1-processing-n:127.0.0.1:40144_solr) [n:127.0.0.1:40144_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42b109da name: ZooKeeperConnection Watcher:127.0.0.1:44351/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 3476238 WARN (zkConnectionManagerCallback-5243-thread-1-processing-n:127.0.0.1:40144_solr) [n:127.0.0.1:40144_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 3476245 WARN (zkConnectionManagerCallback-5241-thread-1-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@55731a08 name: ZooKeeperConnection Watcher:127.0.0.1:44351/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 3476258 WARN (zkConnectionManagerCallback-5241-thread-1-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 3476262 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
[junit4] 2> 3476279 INFO (qtp1265219449-30169) [n:127.0.0.1:63222_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3476279 INFO (qtp1265219449-30169) [n:127.0.0.1:63222_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3476280 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
[junit4] 2> 3476453 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
[junit4] 2> 3476455 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
[junit4] 2> 3476648 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
[junit4] 2> 3476650 INFO (zkConnectionManagerCallback-5252-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3477101 INFO (zkConnectionManagerCallback-5255-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3477102 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3477104 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44351/solr ready
[junit4] 2> 3477148 INFO (qtp1367298367-30175) [n:127.0.0.1:40144_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3477262 INFO (jetty-launcher-5224-thread-2-EventThread) [n:127.0.0.1:63222_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 3477744 INFO (jetty-launcher-5224-thread-1-EventThread) [n:127.0.0.1:40144_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 3478012 INFO (zkConnectionManagerCallback-5243-thread-1-processing-n:127.0.0.1:40144_solr) [n:127.0.0.1:40144_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3478098 INFO (zkConnectionManagerCallback-5241-thread-1-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3478100 INFO (OverseerThreadFactory-10542-thread-1-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection forwardingCollection
[junit4] 2> 3478216 INFO (OverseerStateUpdate-72243918190280709-127.0.0.1:63222_solr-n_0000000000) [n:127.0.0.1:63222_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"forwardingCollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"forwardingCollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:63222/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3478424 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3478424 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3478530 INFO (zkCallback-5240-thread-9-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3478530 INFO (zkCallback-5240-thread-7-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3479478 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3479495 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
[junit4] 2> 3479581 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 3479605 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
[junit4] 2> 3479605 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63222.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d5fb93
[junit4] 2> 3479605 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3479605 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_DC61E2C91E3D03BF-001/tempDir-001/node2/forwardingCollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_DC61E2C91E3D03BF-001/tempDir-001/node2/./forwardingCollection_shard1_replica_n1/data/]
[junit4] 2> 3479607 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=49.009765625, floorSegmentMB=1.884765625, forceMergeDeletesPctAllowed=8.686372130898123, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 3479658 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3479658 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3479660 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3479660 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3479661 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=13, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 3479661 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@99234a7[forwardingCollection_shard1_replica_n1] main]
[junit4] 2> 3479663 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 3479668 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 3479686 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3479686 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593128340286341120
[junit4] 2> 3479686 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 3479692 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 3479692 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 3479692 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 3479692 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 3479693 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 3479693 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 3479693 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 3479694 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 3479695 INFO (searcherExecutor-10547-thread-1-processing-n:127.0.0.1:63222_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@99234a7[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3479702 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3479702 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3479702 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:63222/solr/forwardingCollection_shard1_replica_n1/
[junit4] 2> 3479702 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3479702 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:63222/solr/forwardingCollection_shard1_replica_n1/ has no replicas
[junit4] 2> 3479702 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3479708 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:63222/solr/forwardingCollection_shard1_replica_n1/ shard1
[junit4] 2> 3479810 INFO (zkCallback-5240-thread-9-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3479810 INFO (zkCallback-5240-thread-7-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3479859 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3479863 INFO (qtp1265219449-30166) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1439
[junit4] 2> 3479869 INFO (qtp1367298367-30175) [n:127.0.0.1:40144_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3479964 INFO (zkCallback-5240-thread-7-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3479964 INFO (zkCallback-5240-thread-9-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3480102 INFO (OverseerCollectionConfigSetProcessor-72243918190280709-127.0.0.1:63222_solr-n_0000000000) [n:127.0.0.1:63222_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3480871 INFO (qtp1367298367-30175) [n:127.0.0.1:40144_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3724
[junit4] 2> 3480871 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
[junit4] 2> 3480872 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
[junit4] 2> 3480876 INFO (qtp1265219449-30169) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 3480885 INFO (qtp1367298367-30172) [n:127.0.0.1:40144_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3480892 INFO (qtp1265219449-30169) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 3480892 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[DC61E2C91E3D03BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
[junit4] 2> 3480896 INFO (jetty-closer-5225-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7d2ca322{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3480898 INFO (jetty-closer-5225-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1739678739
[junit4] 2> 3480898 INFO (jetty-closer-5225-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@54d8dbc8{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3480898 INFO (jetty-closer-5225-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3480898 INFO (jetty-closer-5225-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@47f0e463: rootName = solr_40144, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@66b777a0
[junit4] 2> 3480901 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=763429520
[junit4] 2> 3480901 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3480901 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1cb216c7: rootName = solr_63222, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5a73a652
[junit4] 2> 3480905 INFO (jetty-closer-5225-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3480905 INFO (jetty-closer-5225-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7ffcf5b5: rootName = solr_40144, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@616f95f
[junit4] 2> 3480905 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3480905 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@ae5d386: rootName = solr_63222, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@616f95f
[junit4] 2> 3480909 INFO (jetty-closer-5225-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3480909 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3480909 INFO (jetty-closer-5225-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2d56a255: rootName = solr_40144, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@89dcfd5
[junit4] 2> 3480909 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6f65ce65: rootName = solr_63222, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@89dcfd5
[junit4] 2> 3480911 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:63222_solr
[junit4] 2> 3480911 INFO (jetty-closer-5225-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40144_solr
[junit4] 2> 3480912 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3480912 INFO (jetty-closer-5225-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3480912 INFO (zkCallback-5240-thread-9-processing-n:127.0.0.1:63222_solr) [n:127.0.0.1:63222_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 3480912 INFO (coreCloseExecutor-10552-thread-1) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@703c0a64
[junit4] 2> 3480912 INFO (coreCloseExecutor-10552-thread-1) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=1882983012
[junit4] 2> 3480912 INFO (coreCloseExecutor-10552-thread-1) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7f55f9d7: rootName = solr_63222, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@26061875
[junit4] 2> 3480913 INFO (zkCallback-5242-thread-6-processing-n:127.0.0.1:40144_solr) [n:127.0.0.1:40144_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 3480916 ERROR (Thread[Thread-10304,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:40144_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 3480921 INFO (jetty-closer-5225-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@58ceaae{/solr,null,UNAVAILABLE}
[junit4] 2> 3480923 INFO (jetty-closer-5225-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3480930 INFO (coreCloseExecutor-10552-thread-1) [n:127.0.0.1:63222_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=1882983012
[junit4] 2> 3480933 INFO (jetty-closer-5225-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72243918190280709-127.0.0.1:63222_solr-n_0000000000) closing
[junit4] 2> 3480933 INFO (OverseerStateUpdate-72243918190280709-127.0.0.1:63222_solr-n_0000000000) [n:127.0.0.1:63222_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:63222_solr
[junit4] 2> 3480934 WARN (OverseerAutoScalingTriggerThread-72243918190280709-127.0.0.1:63222_solr-n_0000000000) [n:127.0.0.1:63222_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3480943 ERROR (Thread[Thread-10301,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:63222_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 3480952 INFO (jetty-closer-5225-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2773f3d4{/solr,null,UNAVAILABLE}
[junit4] 2> 3480954 INFO (jetty-closer-5225-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3480955 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3480956 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44351 44351
[junit4] 2> 3482369 INFO (Thread-10284) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44351 44351
[junit4] 2> 3482374 WARN (Thread-10284) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/security.json
[junit4] 2> 2 /solr/collections/forwardingCollection/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/collections/forwardingCollection/state.json
[junit4] 2> 4 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
[junit4] 2>
[junit4] 2> 3482397 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[DC61E2C91E3D03BF]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 22, 2018 6:53:03 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> Feb 22, 2018 6:53:23 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] 2> 1) Thread[id=30214, name=jetty-launcher-5224-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] 2> 2) Thread[id=30218, name=jetty-launcher-5224-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] 2> Feb 22, 2018 6:53:23 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=30214, name=jetty-launcher-5224-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> 2) Thread[id=30218, name=jetty-launcher-5224-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> 3502520 ERROR (jetty-launcher-5224-thread-1-EventThread) [n:127.0.0.1:40144_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] 2> 3502523 ERROR (jetty-launcher-5224-thread-2-EventThread) [n:127.0.0.1:63222_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] 2> Feb 22, 2018 6:53:23 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=790, maxMBSortInHeap=6.49105335042505, sim=RandomSimilarity(queryNorm=true): {}, locale=ar-OM, timezone=Etc/GMT+11
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=229792248,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestQueryWrapperFilter, TestPostingsSolrHighlighter, TestPayloadCheckQParserPlugin, TestConfigSets, HdfsDirectoryTest, TestRetrieveFieldsOptimizer, OverseerRolesTest, HdfsBasicDistributedZkTest, RequiredFieldsTest, ReplicationFactorTest, AddSchemaFieldsUpdateProcessorFactoryTest, LeaderElectionIntegrationTest, TestNoOpRegenerator, DistributedVersionInfoTest, BasicDistributedZkTest, TestCollectionAPI, TermVectorComponentDistributedTest, TestPseudoReturnFields, BlockJoinFacetDistribTest, MultiTermTest, TestTrieFacet, SignatureUpdateProcessorFactoryTest, AnalysisAfterCoreReloadTest, MissingSegmentRecoveryTest, TestHdfsCloudBackupRestore, ReplaceNodeNoTargetTest, ZkStateReaderTest, OutputWriterTest, PathHierarchyTokenizerFactoryTest, RequestLoggingTest, TestNodeAddedTrigger, TestShortCircuitedRequests, SolrInfoBeanTest, TestBinaryField, TestSolrCloudWithHadoopAuthPlugin, DistributedQueryComponentCustomSortTest, TestDelegationWithHadoopAuth, TestPointFields, TestManagedSynonymFilterFactory, TestCSVResponseWriter, QueryElevationComponentTest, TestSolrCoreParser, HttpTriggerListenerTest, UniqFieldsUpdateProcessorFactoryTest, TestCloudNestedDocsSort, TestManagedResource, PreAnalyzedUpdateProcessorTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestPhraseSuggestions, AnalyticsMergeStrategyTest, TriLevelCompositeIdRoutingTest, TestUtils, TestEmbeddedSolrServerSchemaAPI, SuggestComponentContextFilterQueryTest, IndexSchemaRuntimeFieldTest, AssignTest, SynonymTokenizerTest, TestTrie, ResourceLoaderTest, TestHashQParserPlugin, HdfsRecoverLeaseTest, TestMissingGroups, TestCustomDocTransformer, DistributedSuggestComponentTest, UpdateLogTest, HdfsChaosMonkeySafeLeaderTest, TestEmbeddedSolrServerAdminHandler, TestDynamicLoading, TestAddFieldRealTimeGet, ShufflingReplicaListTransformerTest, SolrMetricsIntegrationTest, TestFieldTypeResource, TestInfoStreamLogging, CollectionsAPIDistributedZkTest, TestManagedSchemaAPI, ConvertedLegacyTest, NodeAddedTriggerTest, HighlighterMaxOffsetTest, CopyFieldTest, TestExclusionRuleCollectionAccess, TestCoreAdminApis, TestQueryTypes, TestFastOutputStream, TestExceedMaxTermLength, TestCoreContainer, MoreLikeThisHandlerTest, DateMathParserTest, WrapperMergePolicyFactoryTest, TestSSLRandomization, TestCustomStream, RecoveryAfterSoftCommitTest, DirectSolrSpellCheckerTest, FileUtilsTest, RegexBytesRefFilterTest, TestGeoJSONResponseWriter, TestCharFilters, TestGroupingSearch, TestFieldCacheSortRandom, AddReplicaTest, TestStreamBody, CreateCollectionCleanupTest, AtomicUpdatesTest, RankQueryTest, TestPolicyCloud, TestSurroundQueryParser, TestRawResponseWriter, TestFieldCacheWithThreads, EnumFieldTest, TestCustomSort, HDFSCollectionsAPITest, DateFieldTest, TestConfigSetImmutable, SecurityConfHandlerTest, LargeFieldTest, TestCloudSearcherWarming, RollingRestartTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, PluginInfoTest, DeleteInactiveReplicaTest, QueryEqualityTest, CacheHeaderTest, TestNRTOpen, TestRestManager, StatelessScriptUpdateProcessorFactoryTest, ExecutePlanActionTest, TestClusterStateProvider, ConfigSetsAPITest, SampleTest, ConfigureRecoveryStrategyTest, SuggesterTSTTest, ZkFailoverTest, TestLegacyField, TestHighFrequencyDictionaryFactory, TestFileDictionaryLookup, SimpleMLTQParserTest, TestReplicaProperties, TestDFISimilarityFactory, DistribCursorPagingTest, TestInPlaceUpdatesDistrib, DistributedMLTComponentTest, TestZkChroot, SolrJmxReporterTest, TestXIncludeConfig, TestBM25SimilarityFactory, SOLR749Test, TestLRUStatsCache, TestSuggestSpellingConverter, V2StandaloneTest, CoreMergeIndexesAdminHandlerTest, TestFieldResource, TestTolerantUpdateProcessorCloud, TestJavabinTupleStreamParser, TestChildDocTransformer, NumberUtilsTest, TestSearcherReuse, SearchHandlerTest, FastVectorHighlighterTest, TestTrackingShardHandlerFactory, TestFunctionQuery, TestFieldCacheReopen, TestCopyFieldCollectionResource, MetricUtilsTest, TestFoldingMultitermQuery, AssignBackwardCompatibilityTest, SolrIndexMetricsTest, TestCursorMarkWithoutUniqueKey, BinaryUpdateRequestHandlerTest, RecoveryZkTest, TestStandardQParsers, ClassificationUpdateProcessorIntegrationTest, AnalysisErrorHandlingTest, TestRecoveryHdfs, TestSkipOverseerOperations, XmlUpdateRequestHandlerTest, MBeansHandlerTest, TestLeaderElectionZkExpiry, TestDeleteCollectionOnDownNodes, SolrGraphiteReporterTest, TestFaceting, TestJsonFacetRefinement, StressHdfsTest, TestScoreJoinQPNoScore, OverseerTaskQueueTest, TestLegacyNumericUtils, LeaderFailureAfterFreshStartTest, LeaderElectionContextKeyTest, RulesTest, TestRebalanceLeaders, TestTolerantSearch, TestLFUCache, ClusterStateUpdateTest, TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, ZkCLITest, TestDistributedGrouping, TestRecovery, DistributedSpellCheckComponentTest, TestRealTimeGet, TestStressReorder, TestJoin, TestReload, TestStressVersions, DistributedTermsComponentTest, SimpleFacetsTest, SolrCoreTest, SolrCmdDistributorTest, TestSort, TestFiltering, BasicFunctionalityTest, TestIndexSearcher, HighlighterTest, DistributedQueryElevationComponentTest, TestCoreDiscovery, DocValuesTest, TestOmitPositions, DocumentBuilderTest, TermVectorComponentTest, LoggingHandlerTest, SolrPluginUtilsTest, TestCollationField, ReturnFieldsTest, JsonLoaderTest, TestComponentsName, SearchHandlerTest, AlternateDirectoryTest, ResponseLogComponentTest, TestMergePolicyConfig, TestElisionMultitermQuery, ExternalFileFieldSortTest, TestLuceneMatchVersion, EchoParamsTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, ChaosMonkeyNothingIsSafeTest, AsyncCallRequestStatusResponseTest, ChaosMonkeySafeLeaderWithPullReplicasTest, CloudExitableDirectoryReaderTest, CollectionsAPISolrJTest, DeleteShardTest, ForceLeaderTest, LeaderInitiatedRecoveryOnShardRestartTest, LegacyCloudClusterPropTest, MoveReplicaHDFSTest, OverseerCollectionConfigSetProcessorTest, OverseerModifyCollectionTest, PeerSyncReplicationTest, TestCloudPseudoReturnFields, TestCloudRecovery, TestConfigSetsAPIZkFailure, TestDistributedMap, TestDownShardTolerantSearch, TestLeaderElectionWithEmptyReplica, TestLeaderInitiatedRecoveryThread, TestPullReplica, TestSolrCloudWithKerberosAlt, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=DC61E2C91E3D03BF -Dtests.slow=true -Dtests.locale=ar-OM -Dtests.timezone=Etc/GMT+11 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=30214, name=jetty-launcher-5224-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] > 2) Thread[id=30218, name=jetty-launcher-5224-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] > at __randomizedtesting.SeedInfo.seed([DC61E2C91E3D03BF]:0)
[junit4] Completed [567/777 (1!)] on J0 in 55.07s, 12 tests, 1 error <<< FAILURES!
[...truncated 50921 lines...]
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 456 -
Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/456/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.security.hadoop.TestImpersonationWithHadoopAuth
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.security.hadoop.TestImpersonationWithHadoopAuth: 1) Thread[id=9893, name=jetty-launcher-1896-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestImpersonationWithHadoopAuth] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.security.hadoop.TestImpersonationWithHadoopAuth:
1) Thread[id=9893, name=jetty-launcher-1896-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestImpersonationWithHadoopAuth]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
at __randomizedtesting.SeedInfo.seed([664B7BE5FCE5DCCD]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=35909, name=jetty-launcher-8334-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=35909, name=jetty-launcher-8334-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
at __randomizedtesting.SeedInfo.seed([664B7BE5FCE5DCCD]:0)
Build Log:
[...truncated 12740 lines...]
[junit4] Suite: org.apache.solr.security.hadoop.TestImpersonationWithHadoopAuth
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.security.hadoop.TestImpersonationWithHadoopAuth_664B7BE5FCE5DCCD-001/init-core-data-001
[junit4] 2> 1149761 WARN (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 1149761 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1149763 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1149763 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1149801 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.security.hadoop.TestImpersonationWithHadoopAuth_664B7BE5FCE5DCCD-001/tempDir-001
[junit4] 2> 1149801 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1149801 INFO (Thread-3631) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1149801 INFO (Thread-3631) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1149803 ERROR (Thread-3631) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1149905 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:65167
[junit4] 2> 1149914 INFO (zkConnectionManagerCallback-1899-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1149942 INFO (jetty-launcher-1896-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1149944 INFO (jetty-launcher-1896-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1149944 INFO (jetty-launcher-1896-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1149944 INFO (jetty-launcher-1896-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1149944 INFO (jetty-launcher-1896-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c4a5f35{/solr,null,AVAILABLE}
[junit4] 2> 1149945 INFO (jetty-launcher-1896-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2cf24888{HTTP/1.1,[http/1.1]}{127.0.0.1:57442}
[junit4] 2> 1149945 INFO (jetty-launcher-1896-thread-2) [ ] o.e.j.s.Server Started @1153344ms
[junit4] 2> 1149945 INFO (jetty-launcher-1896-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57442}
[junit4] 2> 1149945 ERROR (jetty-launcher-1896-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1149945 INFO (jetty-launcher-1896-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1149945 INFO (jetty-launcher-1896-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1149945 INFO (jetty-launcher-1896-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1149945 INFO (jetty-launcher-1896-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1149945 INFO (jetty-launcher-1896-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T07:49:18.173Z
[junit4] 2> 1149946 INFO (jetty-launcher-1896-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1149946 INFO (jetty-launcher-1896-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1149946 INFO (jetty-launcher-1896-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1149947 INFO (jetty-launcher-1896-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@50ac8a59{/solr,null,AVAILABLE}
[junit4] 2> 1149948 INFO (jetty-launcher-1896-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@fcffb5a{HTTP/1.1,[http/1.1]}{127.0.0.1:60841}
[junit4] 2> 1149948 INFO (jetty-launcher-1896-thread-1) [ ] o.e.j.s.Server Started @1153347ms
[junit4] 2> 1149948 INFO (jetty-launcher-1896-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60841}
[junit4] 2> 1149948 ERROR (jetty-launcher-1896-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1149948 INFO (jetty-launcher-1896-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1149948 INFO (jetty-launcher-1896-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1149948 INFO (jetty-launcher-1896-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1149948 INFO (jetty-launcher-1896-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T07:49:18.176Z
[junit4] 2> 1149950 INFO (zkConnectionManagerCallback-1901-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1149951 INFO (jetty-launcher-1896-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1149952 INFO (zkConnectionManagerCallback-1903-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1149952 INFO (jetty-launcher-1896-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1149982 INFO (jetty-launcher-1896-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65167/solr
[junit4] 2> 1149984 INFO (zkConnectionManagerCallback-1907-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1149997 INFO (jetty-launcher-1896-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65167/solr
[junit4] 2> 1150001 INFO (zkConnectionManagerCallback-1913-thread-1-processing-n:127.0.0.1:60841_solr) [n:127.0.0.1:60841_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1150002 INFO (zkConnectionManagerCallback-1911-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1150015 INFO (zkConnectionManagerCallback-1915-thread-1-processing-n:127.0.0.1:57442_solr) [n:127.0.0.1:57442_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1150055 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1150056 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60841_solr
[junit4] 2> 1150062 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.c.Overseer Overseer (id=72241307093434372-127.0.0.1:60841_solr-n_0000000000) starting
[junit4] 2> 1150075 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60841_solr
[junit4] 2> 1150081 INFO (zkCallback-1912-thread-1-processing-n:127.0.0.1:60841_solr) [n:127.0.0.1:60841_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1150118 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HadoopAuthPlugin
[junit4] 2> 1150122 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.s.HadoopAuthPlugin Params: {proxyuser.anyHostUsersGroup.hosts=*, proxyuser.wrongHost.hosts=1.1.1.1.1.1, proxyuser.bogusGroup.groups=__some_bogus_group, proxyuser.localHostAnyGroup.groups=*, cookie.domain=127.0.0.1, proxyuser.wrongHost.groups=*, type=simple, proxyuser.noHosts.groups=*, proxyuser.anyHostUsersGroup.groups=staff, proxyuser.noGroups.hosts=*, proxyuser.anyHostAnyUser.groups=*, delegation-token.token-kind=solr-dt, proxyuser.bogusGroup.hosts=*, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, signer.secret.provider.zookeeper.path=/token, zk-dt-secret-manager.enable=true, proxyuser.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, proxyuser.anyHostAnyUser.hosts=*, signer.secret.provider=zookeeper}
[junit4] 2> 1150126 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 1150129 INFO (jetty-launcher-1896-thread-1-EventThread) [n:127.0.0.1:60841_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 1150174 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.c.CoreContainer PKIAuthenticationPlugin is managing internode requests
[junit4] 2> 1150196 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1150202 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1150205 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57442_solr
[junit4] 2> 1150206 INFO (zkCallback-1912-thread-1-processing-n:127.0.0.1:60841_solr) [n:127.0.0.1:60841_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1150207 INFO (zkCallback-1914-thread-1-processing-n:127.0.0.1:57442_solr) [n:127.0.0.1:57442_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1150238 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60841.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41538633
[junit4] 2> 1150247 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60841.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41538633
[junit4] 2> 1150247 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60841.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41538633
[junit4] 2> 1150248 INFO (jetty-launcher-1896-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.security.hadoop.TestImpersonationWithHadoopAuth_664B7BE5FCE5DCCD-001/tempDir-001/node2/.
[junit4] 2> 1150269 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HadoopAuthPlugin
[junit4] 2> 1150269 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.s.HadoopAuthPlugin Params: {proxyuser.anyHostUsersGroup.hosts=*, proxyuser.wrongHost.hosts=1.1.1.1.1.1, proxyuser.bogusGroup.groups=__some_bogus_group, proxyuser.localHostAnyGroup.groups=*, cookie.domain=127.0.0.1, proxyuser.wrongHost.groups=*, type=simple, proxyuser.noHosts.groups=*, proxyuser.anyHostUsersGroup.groups=staff, proxyuser.noGroups.hosts=*, proxyuser.anyHostAnyUser.groups=*, delegation-token.token-kind=solr-dt, proxyuser.bogusGroup.hosts=*, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, signer.secret.provider.zookeeper.path=/token, zk-dt-secret-manager.enable=true, proxyuser.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, proxyuser.anyHostAnyUser.hosts=*, signer.secret.provider=zookeeper}
[junit4] 2> 1150271 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 1150282 INFO (jetty-launcher-1896-thread-2-EventThread) [n:127.0.0.1:57442_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 1150341 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.c.CoreContainer PKIAuthenticationPlugin is managing internode requests
[junit4] 2> 1150430 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57442.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41538633
[junit4] 2> 1150437 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57442.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41538633
[junit4] 2> 1150437 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57442.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41538633
[junit4] 2> 1150438 INFO (jetty-launcher-1896-thread-2) [n:127.0.0.1:57442_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.security.hadoop.TestImpersonationWithHadoopAuth_664B7BE5FCE5DCCD-001/tempDir-001/node1/.
[junit4] 2> 1150503 INFO (zkConnectionManagerCallback-1921-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1150508 INFO (zkConnectionManagerCallback-1925-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1150510 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1150511 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:65167/solr ready
[junit4] 2> 1150549 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyInvalidProxyUser-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
[junit4] 2> 1150617 WARN (TEST-TestImpersonationWithHadoopAuth.testProxyInvalidProxyUser-seed#[664B7BE5FCE5DCCD]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=simple-dt&e=1519321758842&s=zHHlHxYBCeb+VcWtQXBVkNMGRyg="; Domain=127.0.0.1; Expires=Kam, 22-Feb-2018 17:49:18 GMT; HttpOnly". Invalid 'expires' attribute: Kam, 22-Feb-2018 17:49:18 GMT
[junit4] 2> 1150617 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyInvalidProxyUser-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
[junit4] 2> 1150709 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyInvalidGroup-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
[junit4] 2> 1150807 WARN (qtp109810651-9847) [ ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
[junit4] 2> id: invalid user name: "bar"
[junit4] 2>
[junit4] 2> 1150810 WARN (TEST-TestImpersonationWithHadoopAuth.testProxyInvalidGroup-seed#[664B7BE5FCE5DCCD]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=simple-dt&e=1519321758949&s=/JhY23TLC/zwhcJKAindSD1rYRY="; Domain=127.0.0.1; Expires=Kam, 22-Feb-2018 17:49:18 GMT; HttpOnly". Invalid 'expires' attribute: Kam, 22-Feb-2018 17:49:18 GMT
[junit4] 2> 1150811 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyInvalidGroup-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
[junit4] 2> 1150845 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyWrongHost-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
[junit4] 2> 1165966 WARN (jetty-launcher-1896-thread-2-SendThread(127.0.0.1:65167)) [n:127.0.0.1:57442_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 15532ms for sessionid 0x100a716076a0006
[junit4] 2> 1165966 WARN (jetty-launcher-1896-thread-1-SendThread(127.0.0.1:65167)) [n:127.0.0.1:60841_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 15594ms for sessionid 0x100a716076a0007
[junit4] 2> 1165967 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a716076a0006, likely client has closed socket
[junit4] 2> 1165967 WARN (jetty-launcher-1896-thread-1-SendThread(127.0.0.1:65167)) [n:127.0.0.1:60841_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 15665ms for sessionid 0x100a716076a0004
[junit4] 2> 1165967 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a716076a0007, likely client has closed socket
[junit4] 2> 1165966 WARN (jetty-launcher-1896-thread-2-SendThread(127.0.0.1:65167)) [n:127.0.0.1:57442_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 15595ms for sessionid 0x100a716076a0008
[junit4] 2> 1165967 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a716076a0004, likely client has closed socket
[junit4] 2> 1165968 WARN (TEST-TestImpersonationWithHadoopAuth.testProxyWrongHost-seed#[664B7BE5FCE5DCCD]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=simple-dt&e=1519321759075&s=ngiTx5X0FbK3aQCAeNZIBdL8ykk="; Domain=127.0.0.1; Expires=Kam, 22-Feb-2018 17:49:19 GMT; HttpOnly". Invalid 'expires' attribute: Kam, 22-Feb-2018 17:49:19 GMT
[junit4] 2> 1165968 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a716076a0008, likely client has closed socket
[junit4] 2> 1165968 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyWrongHost-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
[junit4] 2> 1165995 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyValidateHost-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
[junit4] 2> 1165997 INFO (qtp109810651-9847) [n:127.0.0.1:60841_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&user.name=localHostAnyGroup&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1165997 INFO (qtp109810651-9847) [n:127.0.0.1:60841_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&user.name=localHostAnyGroup&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1165998 WARN (TEST-TestImpersonationWithHadoopAuth.testProxyValidateHost-seed#[664B7BE5FCE5DCCD]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=simple-dt&e=1519321774225&s=FOniuiMETM7RomhNhPN8/RHowkw="; Domain=127.0.0.1; Expires=Kam, 22-Feb-2018 17:49:34 GMT; HttpOnly". Invalid 'expires' attribute: Kam, 22-Feb-2018 17:49:34 GMT
[junit4] 2> 1165998 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyValidateHost-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
[junit4] 2> 1166019 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyValidateGroup-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
[junit4] 2> 1166021 INFO (qtp109810651-9847) [n:127.0.0.1:60841_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&user.name=anyHostUsersGroup&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1166021 INFO (qtp109810651-9847) [n:127.0.0.1:60841_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&user.name=anyHostUsersGroup&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1166022 WARN (TEST-TestImpersonationWithHadoopAuth.testProxyValidateGroup-seed#[664B7BE5FCE5DCCD]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=simple-dt&e=1519321774249&s=HEVRyJ1DfHaeauYGC8pS5NtPsMQ="; Domain=127.0.0.1; Expires=Kam, 22-Feb-2018 17:49:34 GMT; HttpOnly". Invalid 'expires' attribute: Kam, 22-Feb-2018 17:49:34 GMT
[junit4] 2> 1166022 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyValidateGroup-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
[junit4] 2> 1166043 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyNoConfigGroups-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
[junit4] 2> 1166047 WARN (TEST-TestImpersonationWithHadoopAuth.testProxyNoConfigGroups-seed#[664B7BE5FCE5DCCD]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=simple-dt&e=1519321774273&s=BQwWMtnuL5NKvY2JalbRvYz/Uns="; Domain=127.0.0.1; Expires=Kam, 22-Feb-2018 17:49:34 GMT; HttpOnly". Invalid 'expires' attribute: Kam, 22-Feb-2018 17:49:34 GMT
[junit4] 2> 1166047 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyNoConfigGroups-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
[junit4] IGNOR/A 0.00s J1 | TestImpersonationWithHadoopAuth.testForwarding
[junit4] > Assumption #1: 'awaitsfix' test group is disabled (@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/HADOOP-9893))
[junit4] 2> 1166067 WARN (zkConnectionManagerCallback-1915-thread-1-processing-n:127.0.0.1:57442_solr) [n:127.0.0.1:57442_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f230156 name: ZooKeeperConnection Watcher:127.0.0.1:65167/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 1166068 WARN (zkConnectionManagerCallback-1915-thread-1-processing-n:127.0.0.1:57442_solr) [n:127.0.0.1:57442_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 1166070 WARN (zkConnectionManagerCallback-1913-thread-1-processing-n:127.0.0.1:60841_solr) [n:127.0.0.1:60841_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1df4f30d name: ZooKeeperConnection Watcher:127.0.0.1:65167/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 1166070 WARN (zkConnectionManagerCallback-1913-thread-1-processing-n:127.0.0.1:60841_solr) [n:127.0.0.1:60841_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 1166072 WARN (zkConnectionManagerCallback-1925-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b946a48 name: ZooKeeperConnection Watcher:127.0.0.1:65167/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 1166072 WARN (zkConnectionManagerCallback-1925-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 1166074 INFO (jetty-launcher-1896-thread-2-EventThread) [n:127.0.0.1:57442_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 1166082 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyNullProxyUser-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
[junit4] 2> 1166084 INFO (qtp109810651-9847) [n:127.0.0.1:60841_solr ] o.a.s.s.SolrDispatchFilter Error authenticating
[junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:195)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.HadoopAuthFilter.doFilter(HadoopAuthFilter.java:110)
[junit4] 2> at org.apache.solr.security.HadoopAuthPlugin.doAuthenticate(HadoopAuthPlugin.java:245)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:454)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1166085 WARN (qtp109810651-9847) [n:127.0.0.1:60841_solr ] o.e.j.s.HttpChannel /solr/admin/collections
[junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:460)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:195)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.HadoopAuthFilter.doFilter(HadoopAuthFilter.java:110)
[junit4] 2> at org.apache.solr.security.HadoopAuthPlugin.doAuthenticate(HadoopAuthPlugin.java:245)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:454)
[junit4] 2> ... 31 more
[junit4] 2> 1166091 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyNullProxyUser-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
[junit4] 2> 1166116 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyValidateAnyHostAnyUser-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
[junit4] 2> 1166118 INFO (qtp109810651-9843) [n:127.0.0.1:60841_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&user.name=anyHostAnyUser&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1166118 INFO (qtp109810651-9843) [n:127.0.0.1:60841_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&user.name=anyHostAnyUser&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1166118 WARN (TEST-TestImpersonationWithHadoopAuth.testProxyValidateAnyHostAnyUser-seed#[664B7BE5FCE5DCCD]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=simple-dt&e=1519321774345&s=IccpRisAzOxzK6Ul30CjJqsGR74="; Domain=127.0.0.1; Expires=Kam, 22-Feb-2018 17:49:34 GMT; HttpOnly". Invalid 'expires' attribute: Kam, 22-Feb-2018 17:49:34 GMT
[junit4] 2> 1166118 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyValidateAnyHostAnyUser-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
[junit4] 2> 1166142 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyNoConfigHosts-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
[junit4] 2> 1166146 WARN (TEST-TestImpersonationWithHadoopAuth.testProxyNoConfigHosts-seed#[664B7BE5FCE5DCCD]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=simple-dt&e=1519321774372&s=55BeW3BTxVr8rdP1tQr3XTDJnUI="; Domain=127.0.0.1; Expires=Kam, 22-Feb-2018 17:49:34 GMT; HttpOnly". Invalid 'expires' attribute: Kam, 22-Feb-2018 17:49:34 GMT
[junit4] 2> 1166146 INFO (TEST-TestImpersonationWithHadoopAuth.testProxyNoConfigHosts-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
[junit4] 2> 1167142 INFO (jetty-closer-1897-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@fcffb5a{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1167142 INFO (jetty-closer-1897-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2cf24888{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1167143 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=183697033
[junit4] 2> 1167143 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=287571485
[junit4] 2> 1167143 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1167143 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4036d69f: rootName = solr_60841, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@4f2914ab
[junit4] 2> 1167143 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1167143 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2e2b551f: rootName = solr_57442, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7499b9fe
[junit4] 2> 1167148 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1167148 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@78991771: rootName = solr_60841, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2b4611d4
[junit4] 2> 1167148 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1167148 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@65b249a: rootName = solr_57442, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2b4611d4
[junit4] 2> 1167150 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1167150 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@144e693c: rootName = solr_60841, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@94ee25a
[junit4] 2> 1167150 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1167151 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5ca57bdc: rootName = solr_57442, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@94ee25a
[junit4] 2> 1167244 INFO (zkConnectionManagerCallback-1913-thread-1-processing-n:127.0.0.1:60841_solr) [n:127.0.0.1:60841_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1167245 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:60841_solr
[junit4] 2> 1167246 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1167246 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1167246 INFO (jetty-closer-1897-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72241307093434372-127.0.0.1:60841_solr-n_0000000000) closing
[junit4] 2> 1167247 INFO (OverseerStateUpdate-72241307093434372-127.0.0.1:60841_solr-n_0000000000) [n:127.0.0.1:60841_solr ] o.a.s.c.Overseer According to ZK I (id=72241307093434372-127.0.0.1:60841_solr-n_0000000000) am no longer a leader.
[junit4] 2> 1167247 INFO (OverseerStateUpdate-72241307093434372-127.0.0.1:60841_solr-n_0000000000) [n:127.0.0.1:60841_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60841_solr
[junit4] 2> 1167248 INFO (OverseerCollectionConfigSetProcessor-72241307093434372-127.0.0.1:60841_solr-n_0000000000) [n:127.0.0.1:60841_solr ] o.a.s.c.OverseerTaskProcessor According to ZK I (id=72241307093434372-127.0.0.1:60841_solr-n_0000000000) am no longer a leader.
[junit4] 2> 1167249 WARN (OverseerAutoScalingTriggerThread-72241307093434372-127.0.0.1:60841_solr-n_0000000000) [n:127.0.0.1:60841_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1167250 INFO (zkCallback-1912-thread-5-processing-n:127.0.0.1:60841_solr) [n:127.0.0.1:60841_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1167250 WARN (ScheduledTrigger-3227-thread-1) [n:127.0.0.1:60841_solr ] o.a.s.c.a.TriggerBase Exception updating trigger state '/autoscaling/triggerState/.auto_add_replicas'
[junit4] 2> java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316)
[junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:58)
[junit4] 2> at org.apache.solr.cloud.autoscaling.TriggerBase.saveState(TriggerBase.java:203)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.run(ScheduledTriggers.java:573)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1167253 ERROR (Thread[Thread-3648,5,TGRP-TestImpersonationWithHadoopAuth]) [n:127.0.0.1:60841_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 1167612 INFO (jetty-closer-1897-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@50ac8a59{/solr,null,UNAVAILABLE}
[junit4] 2> 1167613 INFO (jetty-closer-1897-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1167820 INFO (zkConnectionManagerCallback-1915-thread-1-processing-n:127.0.0.1:57442_solr) [n:127.0.0.1:57442_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1167821 INFO (zkCallback-1914-thread-5-processing-n:127.0.0.1:57442_solr) [n:127.0.0.1:57442_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1167821 INFO (zkCallback-1914-thread-1-processing-n:127.0.0.1:57442_solr) [n:127.0.0.1:57442_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57442_solr
[junit4] 2> 1167822 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:57442_solr
[junit4] 2> 1167823 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1167823 INFO (jetty-closer-1897-thread-2) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1167823 INFO (zkCallback-1914-thread-5-processing-n:127.0.0.1:57442_solr) [n:127.0.0.1:57442_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1167825 ERROR (Thread[Thread-3651,5,TGRP-TestImpersonationWithHadoopAuth]) [n:127.0.0.1:57442_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 1168055 INFO (jetty-closer-1897-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2c4a5f35{/solr,null,UNAVAILABLE}
[junit4] 2> 1168056 INFO (jetty-closer-1897-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1168057 ERROR (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1168059 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:65167 65167
[junit4] 2> 1173159 INFO (Thread-3631) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:65167 65167
[junit4] 2> 1173160 WARN (Thread-3631) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
[junit4] 2>
[junit4] 2> 1173160 INFO (SUITE-TestImpersonationWithHadoopAuth-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 22, 2018 7:49:41 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 22, 2018 7:50:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.security.hadoop.TestImpersonationWithHadoopAuth:
[junit4] 2> 1) Thread[id=9893, name=jetty-launcher-1896-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestImpersonationWithHadoopAuth]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] 2> Feb 22, 2018 7:50:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=9893, name=jetty-launcher-1896-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestImpersonationWithHadoopAuth]
[junit4] 2> 1193258 ERROR (jetty-launcher-1896-thread-2-EventThread) [n:127.0.0.1:57442_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] 2> Feb 22, 2018 7:50:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1203, maxMBSortInHeap=5.987098023249644, sim=RandomSimilarity(queryNorm=true): {}, locale=id, timezone=US/Aleutian
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=30673680,total=427352064
[junit4] 2> NOTE: All tests run in this JVM: [JsonLoaderTest, TestCrossCoreJoin, TestSubQueryTransformer, TestCoreDiscovery, TestDynamicFieldCollectionResource, TestWordDelimiterFilterFactory, TestOverriddenPrefixQueryForCustomFieldType, TestDirectoryFactory, UpdateParamsTest, TestRandomDVFaceting, TestRTGBase, TestManagedSynonymGraphFilterFactory, TestSizeLimitedDistributedMap, DistanceUnitsTest, TestIndexingPerformance, TestExactStatsCache, TestDFRSimilarityFactory, BlobRepositoryCloudTest, HdfsRecoveryZkTest, ReplicaListTransformerTest, CoreSorterTest, TestLegacyFieldReuse, TestQuerySenderListener, VMParamsZkACLAndCredentialsProvidersTest, TestPayloadScoreQParserPlugin, UninvertDocValuesMergePolicyTest, TestCloudInspectUtil, TestFastWriter, URLClassifyProcessorTest, TestCSVLoader, FullSolrCloudDistribCmdsTest, TestInPlaceUpdatesStandalone, SubstringBytesRefFilterTest, EchoParamsTest, DeleteLastCustomShardedReplicaTest, TestSha256AuthenticationProvider, SchemaApiFailureTest, TestTlogReplica, TestConfigSetsAPIExclusivity, SuggesterWFSTTest, TestIntervalFaceting, FieldAnalysisRequestHandlerTest, ZkStateWriterTest, TestReloadDeadlock, TestDocSet, TestFieldSortValues, TestSimpleQParserPlugin, HdfsChaosMonkeyNothingIsSafeTest, AutoAddReplicasPlanActionTest, DOMUtilTest, SuggesterTest, ChaosMonkeySafeLeaderWithPullReplicasTest, SuggestComponentTest, TestSolrFieldCacheBean, TestFieldCacheSort, TestCollapseQParserPlugin, TestMacroExpander, RuleEngineTest, CheckHdfsIndexTest, NodeLostTriggerTest, DocExpirationUpdateProcessorFactoryTest, BooleanFieldTest, AnalyticsQueryTest, MetricTriggerTest, AddBlockUpdateTest, ClassificationUpdateProcessorFactoryTest, DocValuesTest, SolrCoreCheckLockOnStartupTest, TestConfigSetProperties, TestJettySolrRunner, MigrateRouteKeyTest, TestConfigsApi, V2ApiIntegrationTest, OverseerTest, TestReRankQParserPlugin, TestNamedUpdateProcessors, TestStressRecovery, TestNumericTokenStream, SolrSlf4jReporterTest, BlockJoinFacetRandomTest, SolrCmdDistributorTest, CollectionStateFormat2Test, TestRTimerTree, TestAtomicUpdateErrorCases, TermVectorComponentTest, PrimUtilsTest, ResponseHeaderTest, TestSolrCloudWithDelegationTokens, CdcrVersionReplicationTest, TestFieldTypeCollectionResource, SpellPossibilityIteratorTest, TestExportWriter, ComputePlanActionTest, CircularListTest, HdfsAutoAddReplicasIntegrationTest, TestSolrCoreProperties, TestLegacyFieldCache, TestImpersonationWithHadoopAuth]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestImpersonationWithHadoopAuth -Dtests.seed=664B7BE5FCE5DCCD -Dtests.slow=true -Dtests.locale=id -Dtests.timezone=US/Aleutian -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestImpersonationWithHadoopAuth (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.security.hadoop.TestImpersonationWithHadoopAuth:
[junit4] > 1) Thread[id=9893, name=jetty-launcher-1896-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestImpersonationWithHadoopAuth]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] > at __randomizedtesting.SeedInfo.seed([664B7BE5FCE5DCCD]:0)
[junit4] Completed [223/777 (1!)] on J1 in 43.51s, 10 tests, 1 error, 1 skipped <<< FAILURES!
[...truncated 1116 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_664B7BE5FCE5DCCD-001/init-core-data-001
[junit4] 2> 3500155 WARN (SUITE-TestSolrCloudWithSecureImpersonation-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
[junit4] 2> 3500156 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3500157 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 3500157 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 3500186 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_664B7BE5FCE5DCCD-001/tempDir-001
[junit4] 2> 3500186 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3500187 INFO (Thread-9060) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3500187 INFO (Thread-9060) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3500188 ERROR (Thread-9060) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3500287 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[664B7BE5FCE5DCCD]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:44552
[junit4] 2> 3500289 INFO (zkConnectionManagerCallback-8337-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3500294 INFO (jetty-launcher-8334-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3500294 INFO (jetty-launcher-8334-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3500313 INFO (jetty-launcher-8334-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3500313 INFO (jetty-launcher-8334-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3500313 INFO (jetty-launcher-8334-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 3500313 INFO (jetty-launcher-8334-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3500313 INFO (jetty-launcher-8334-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3500313 INFO (jetty-launcher-8334-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 3500313 INFO (jetty-launcher-8334-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@75efc18{/solr,null,AVAILABLE}
[junit4] 2> 3500313 INFO (jetty-launcher-8334-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4e70b88d{/solr,null,AVAILABLE}
[junit4] 2> 3500314 INFO (jetty-launcher-8334-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@47983774{SSL,[ssl, http/1.1]}{127.0.0.1:58744}
[junit4] 2> 3500314 INFO (jetty-launcher-8334-thread-1) [ ] o.e.j.s.Server Started @3506852ms
[junit4] 2> 3500314 INFO (jetty-launcher-8334-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58744}
[junit4] 2> 3500314 ERROR (jetty-launcher-8334-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3500314 INFO (jetty-launcher-8334-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 3500314 INFO (jetty-launcher-8334-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3500314 INFO (jetty-launcher-8334-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3500314 INFO (jetty-launcher-8334-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T08:28:28.660Z
[junit4] 2> 3500315 INFO (jetty-launcher-8334-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2a89fbb6{SSL,[ssl, http/1.1]}{127.0.0.1:45966}
[junit4] 2> 3500315 INFO (jetty-launcher-8334-thread-2) [ ] o.e.j.s.Server Started @3506854ms
[junit4] 2> 3500315 INFO (jetty-launcher-8334-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45966}
[junit4] 2> 3500316 ERROR (jetty-launcher-8334-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3500316 INFO (jetty-launcher-8334-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 3500316 INFO (jetty-launcher-8334-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3500316 INFO (jetty-launcher-8334-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3500316 INFO (jetty-launcher-8334-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T08:28:28.662Z
[junit4] 2> 3500318 INFO (zkConnectionManagerCallback-8339-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3500318 INFO (zkConnectionManagerCallback-8341-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3500318 INFO (jetty-launcher-8334-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3500318 INFO (jetty-launcher-8334-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3500327 INFO (jetty-launcher-8334-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44552/solr
[junit4] 2> 3500327 INFO (jetty-launcher-8334-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44552/solr
[junit4] 2> 3500328 INFO (zkConnectionManagerCallback-8349-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3500329 INFO (zkConnectionManagerCallback-8347-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3500331 INFO (zkConnectionManagerCallback-8352-thread-1-processing-n:127.0.0.1:58744_solr) [n:127.0.0.1:58744_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3500332 INFO (zkConnectionManagerCallback-8353-thread-1-processing-n:127.0.0.1:45966_solr) [n:127.0.0.1:45966_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3500439 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3500440 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45966_solr
[junit4] 2> 3500441 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.c.Overseer Overseer (id=72241461333852166-127.0.0.1:45966_solr-n_0000000000) starting
[junit4] 2> 3500450 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45966_solr
[junit4] 2> 3500455 INFO (OverseerStateUpdate-72241461333852166-127.0.0.1:45966_solr-n_0000000000) [n:127.0.0.1:45966_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3500516 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 3500525 WARN (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3500533 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 3500543 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3500579 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3500585 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58744_solr
[junit4] 2> 3500587 INFO (zkCallback-8351-thread-1-processing-n:127.0.0.1:45966_solr) [n:127.0.0.1:45966_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3500588 INFO (zkCallback-8350-thread-1-processing-n:127.0.0.1:58744_solr) [n:127.0.0.1:58744_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3500613 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 3500629 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 3500629 WARN (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3500630 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 3500644 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 3500670 INFO (jetty-launcher-8334-thread-1-EventThread) [n:127.0.0.1:58744_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 3500670 INFO (jetty-launcher-8334-thread-2-EventThread) [n:127.0.0.1:45966_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 3501793 WARN (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3501794 WARN (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3501839 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45966.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6329caff
[junit4] 2> 3501844 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58744.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6329caff
[junit4] 2> 3501848 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45966.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6329caff
[junit4] 2> 3501849 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45966.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6329caff
[junit4] 2> 3501850 INFO (jetty-launcher-8334-thread-2) [n:127.0.0.1:45966_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_664B7BE5FCE5DCCD-001/tempDir-001/node2/.
[junit4] 2> 3501854 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58744.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6329caff
[junit4] 2> 3501854 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58744.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6329caff
[junit4] 2> 3501855 INFO (jetty-launcher-8334-thread-1) [n:127.0.0.1:58744_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_664B7BE5FCE5DCCD-001/tempDir-001/node1/.
[junit4] 2> 3501903 INFO (zkConnectionManagerCallback-8359-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3501932 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
[junit4] 2> 3502002 WARN (qtp1999082171-35861) [ ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
[junit4] 2> id: invalid user name: "bar"
[junit4] 2>
[junit4] 2> 3502021 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[664B7BE5FCE5DCCD]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1519324110314&s=m2HrKJsFIwmZA/sERLK5XPJaUis="; Path=/; Domain=127.0.0.1; Expires= , 22- -2018 18:28:30 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 22- -2018 18:28:30 GMT
[junit4] 2> 3502021 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
[junit4] 2> 3502046 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[664B7BE5FCE5DCCD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
[junit4] 2> 3502046 INFO (qtp1999082171-35864) [ ] o.a.s.s.SolrDispatchFilter Error authenticating
[junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:454)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:289)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:149)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3502048 WARN (qtp1999082171-35864) [ ] o.e.j.s.HttpChannel /solr/admin/collections
[junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:460)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler
[...truncated too long message...]
org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] 2> Feb 22, 2018 8:29:15 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=30019, maxDocsPerChunk=129, blockSize=224), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=30019, blockSize=224)), sim=RandomSimilarity(queryNorm=true): {}, locale=ar-MA, timezone=America/Argentina/Catamarca
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=258459320,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [NumericFieldsTest, ChangedSchemaMergeTest, TestFieldCacheSortRandom, TestCustomStream, TestTriggerIntegration, UpdateLogTest, TestBM25SimilarityFactory, SolrIndexConfigTest, TestTolerantSearch, TestTrieFacet, TestGenericDistributedQueue, DistribCursorPagingTest, TestRebalanceLeaders, TestSortByMinMaxFunction, StatelessScriptUpdateProcessorFactoryTest, TestPolicyCloud, TestSearcherReuse, DistributedQueryElevationComponentTest, TestSystemCollAutoCreate, TestSurroundQueryParser, SolrCLIZkUtilsTest, TestJsonRequest, TestConfigSetsAPI, TestRawResponseWriter, XmlUpdateRequestHandlerTest, TestExactSharedStatsCache, TestFiltering, ZkFailoverTest, UUIDUpdateProcessorFallbackTest, TestMinMaxOnMultiValuedField, TestFieldResource, TestMultiValuedNumericRangeQuery, DirectoryFactoryTest, OverseerRolesTest, TestXIncludeConfig, HttpSolrCallGetCoreTest, TestIBSimilarityFactory, BitVectorTest, ImplicitSnitchTest, HdfsDirectoryTest, TestPayloadCheckQParserPlugin, CopyFieldTest, TestCopyFieldCollectionResource, TestFaceting, OpenExchangeRatesOrgProviderTest, TestCustomDocTransformer, TestSolrCloudSnapshots, DistributedSuggestComponentTest, TestUninvertingReader, SuggestComponentContextFilterQueryTest, CollectionTooManyReplicasTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestHashQParserPlugin, TestAddFieldRealTimeGet, RankQueryTest, LeaderElectionContextKeyTest, TestDistributedStatsComponentCardinality, TestStandardQParsers, TestAnalyzeInfixSuggestions, BasicDistributedZkTest, TestUtils, TestCollectionAPI, AddSchemaFieldsUpdateProcessorFactoryTest, TestNumericRangeQuery64, CreateCollectionCleanupTest, BasicZkTest, DistributedFacetExistsSmallTest, TestDeleteCollectionOnDownNodes, HdfsThreadLeakTest, TestShortCircuitedRequests, TestRuleBasedAuthorizationPlugin, TestQueryUtils, TestLegacyNumericUtils, CustomCollectionTest, TestLeaderElectionZkExpiry, TestUtilizeNode, StressHdfsTest, TestConfigSetsAPIZkFailure, TestCSVResponseWriter, TestSuggestSpellingConverter, FastVectorHighlighterTest, TestCollectionAPIs, TestBinaryField, CdcrBidirectionalTest, StatsReloadRaceTest, PolyFieldTest, AnalysisAfterCoreReloadTest, TestNoOpRegenerator, TestSmileRequest, SolrMetricManagerTest, TestLegacyNumericRangeQueryBuilder, DateRangeFieldTest, DirectSolrSpellCheckerTest, TriLevelCompositeIdRoutingTest, TestSchemalessBufferedUpdates, LegacyCloudClusterPropTest, TestLegacyField, OpenCloseCoreStressTest, TestTrackingShardHandlerFactory, SpatialFilterTest, BlockDirectoryTest, HLLUtilTest, TestHalfAndHalfDocValues, RegexBytesRefFilterTest, SolrRequestParserTest, TestShardHandlerFactory, HighlighterMaxOffsetTest, ClassificationUpdateProcessorIntegrationTest, TestFilteredDocIdSet, DataDrivenBlockJoinTest, AutoScalingHandlerTest, SolrJmxReporterTest, TestReload, ConvertedLegacyTest, CdcrRequestHandlerTest, TestSolrConfigHandler, TestConfigSetImmutable, SolrGraphiteReporterTest, TestInPlaceUpdatesDistrib, LargeFieldTest, TestOmitPositions, DirectSolrConnectionTest, PreAnalyzedUpdateProcessorTest, SecurityConfHandlerTest, RollingRestartTest, TestCursorMarkWithoutUniqueKey, TestRecoveryHdfs, TestJoin, TestGroupingSearch, TestQuerySenderNoQuery, SuggesterFSTTest, AnalyticsMergeStrategyTest, SolrMetricsIntegrationTest, AtomicUpdatesTest, TestSSLRandomization, TestStressCloudBlindAtomicUpdates, PathHierarchyTokenizerFactoryTest, ZkNodePropsTest, CacheHeaderTest, AddReplicaTest, ConjunctionSolrSpellCheckerTest, TestEmbeddedSolrServerSchemaAPI, V2StandaloneTest, TestHighlightDedupGrouping, TestDynamicFieldResource, TestExceedMaxTermLength, StatsComponentTest, TestCloudDeleteByQuery, TestMissingGroups, HdfsUnloadDistributedZkTest, CurrencyRangeFacetCloudTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, SpellingQueryConverterTest, TestElisionMultitermQuery, FullHLLTest, RequiredFieldsTest, TestJavabinTupleStreamParser, ConfigureRecoveryStrategyTest, ResourceLoaderTest, TestRandomFlRTGCloud, QueryElevationComponentTest, TestSweetSpotSimilarityFactory, TestObjectReleaseTracker, BasicAuthStandaloneTest, TestClusterStateProvider, TestRecovery, TestSchemaVersionResource, TestQueryTypes, TestDynamicLoading, CoreAdminCreateDiscoverTest, WordBreakSolrSpellCheckerTest, TestSolrConfigHandlerCloud, TestOnReconnectListenerSupport, TestJmxIntegration, TestPseudoReturnFields, TestCloudPseudoReturnFields, TestGeoJSONResponseWriter, MultiTermTest, SolrInfoBeanTest, TestStressLiveNodes, LeaderElectionTest, TermsComponentTest, TestQueryWrapperFilter, TestFastOutputStream, TestEmbeddedSolrServerAdminHandler, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, ShardRoutingTest, RecoveryZkTest, TestReplicationHandler, ClusterStateUpdateTest, TestRandomFaceting, TestZkChroot, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestStressReorder, TestStressVersions, DistributedTermsComponentTest, TestRangeQuery, TestCoreContainer, TestSolr4Spatial, SolrCoreTest, TestSort, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, TestIndexSearcher, HighlighterTest, SoftAutoCommitTest, TestLFUCache, TestSolrIndexConfig, BadComponentTest, TestMergePolicyConfig, OutputWriterTest, TestLuceneMatchVersion, TestCharFilters, TestLMJelinekMercerSimilarityFactory, TestFastLRUCache, PreAnalyzedFieldTest, ClusterStateTest, SliceStateTest, UUIDFieldTest, CursorPagingTest, SolrTestCaseJ4Test, TestDistributedMissingSort, TestLuceneIndexBackCompat, ActionThrottleTest, AliasIntegrationTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, CollectionsAPISolrJTest, CreateRoutedAliasTest, DeleteReplicaTest, DistributedVersionInfoTest, HttpPartitionOnCommitTest, HttpPartitionTest, LIRRollingUpdatesTest, LeaderInitiatedRecoveryOnShardRestartTest, MissingSegmentRecoveryTest, MoveReplicaHDFSFailoverTest, MoveReplicaTest, MultiThreadedOCPTest, RecoveryAfterSoftCommitTest, ReplaceNodeNoTargetTest, RestartWhileUpdatingTest, SharedFSAutoReplicaFailoverTest, SolrCloudExampleTest, TestCloudRecovery, TestClusterProperties, TestCryptoKeys, TestDownShardTolerantSearch, TestLeaderElectionWithEmptyReplica, TestPrepRecovery, TestSegmentSorting, TestSkipOverseerOperations, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=664B7BE5FCE5DCCD -Dtests.slow=true -Dtests.locale=ar-MA -Dtests.timezone=America/Argentina/Catamarca -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=35909, name=jetty-launcher-8334-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] > at __randomizedtesting.SeedInfo.seed([664B7BE5FCE5DCCD]:0)
[junit4] Completed [578/777 (2!)] on J0 in 47.33s, 12 tests, 1 error <<< FAILURES!
[...truncated 49362 lines...]
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 455 -
Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/455/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=31168, name=jetty-launcher-6042-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 2) Thread[id=31083, name=jetty-launcher-6042-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) 3) Thread[id=31167, name=jetty-launcher-6042-thread-1-SendThread(127.0.0.1:38874), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=31168, name=jetty-launcher-6042-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
2) Thread[id=31083, name=jetty-launcher-6042-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
3) Thread[id=31167, name=jetty-launcher-6042-thread-1-SendThread(127.0.0.1:38874), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
at __randomizedtesting.SeedInfo.seed([74BFBD089602893A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=31167, name=jetty-launcher-6042-thread-1-SendThread(127.0.0.1:38874), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=31167, name=jetty-launcher-6042-thread-1-SendThread(127.0.0.1:38874), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
at __randomizedtesting.SeedInfo.seed([74BFBD089602893A]:0)
Build Log:
[...truncated 14039 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_74BFBD089602893A-001/init-core-data-001
[junit4] 2> 3513837 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[74BFBD089602893A]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3513838 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[74BFBD089602893A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 3513838 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[74BFBD089602893A]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 3513864 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[74BFBD089602893A]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_74BFBD089602893A-001/tempDir-001
[junit4] 2> 3513864 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[74BFBD089602893A]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3513865 INFO (Thread-6416) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3513865 INFO (Thread-6416) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3513867 ERROR (Thread-6416) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3513964 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[74BFBD089602893A]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:38874
[junit4] 2> 3513967 INFO (zkConnectionManagerCallback-6045-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3513971 INFO (jetty-launcher-6042-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3513971 INFO (jetty-launcher-6042-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3513972 INFO (jetty-launcher-6042-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3513972 INFO (jetty-launcher-6042-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3513972 INFO (jetty-launcher-6042-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 3513973 INFO (jetty-launcher-6042-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3513973 INFO (jetty-launcher-6042-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3513973 INFO (jetty-launcher-6042-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 3513973 INFO (jetty-launcher-6042-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7cb963{/solr,null,AVAILABLE}
[junit4] 2> 3513973 INFO (jetty-launcher-6042-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c983469{/solr,null,AVAILABLE}
[junit4] 2> 3513975 INFO (jetty-launcher-6042-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6f5e44f1{HTTP/1.1,[http/1.1]}{127.0.0.1:53775}
[junit4] 2> 3513975 INFO (jetty-launcher-6042-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7e616ef2{HTTP/1.1,[http/1.1]}{127.0.0.1:57084}
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-2) [ ] o.e.j.s.Server Started @3520729ms
[junit4] 2> 3513975 INFO (jetty-launcher-6042-thread-1) [ ] o.e.j.s.Server Started @3520729ms
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53775}
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57084}
[junit4] 2> 3513976 ERROR (jetty-launcher-6042-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3513976 ERROR (jetty-launcher-6042-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-21T11:35:54.279Z
[junit4] 2> 3513976 INFO (jetty-launcher-6042-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-21T11:35:54.279Z
[junit4] 2> 3513979 INFO (zkConnectionManagerCallback-6049-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3513979 INFO (zkConnectionManagerCallback-6048-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3513979 INFO (jetty-launcher-6042-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3513979 INFO (jetty-launcher-6042-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3513983 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a2bdb0910001, likely client has closed socket
[junit4] 2> 3513985 INFO (jetty-launcher-6042-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38874/solr
[junit4] 2> 3513986 INFO (jetty-launcher-6042-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38874/solr
[junit4] 2> 3513987 INFO (zkConnectionManagerCallback-6055-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3513987 INFO (zkConnectionManagerCallback-6057-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3513990 INFO (zkConnectionManagerCallback-6059-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3513990 INFO (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3514136 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3514137 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53775_solr
[junit4] 2> 3514138 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.c.Overseer Overseer (id=72236529632739333-127.0.0.1:53775_solr-n_0000000000) starting
[junit4] 2> 3514140 INFO (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3514142 INFO (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57084_solr
[junit4] 2> 3514143 INFO (zkCallback-6058-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3514144 INFO (zkCallback-6060-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3514154 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53775_solr
[junit4] 2> 3514156 INFO (zkCallback-6060-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3514156 INFO (zkCallback-6058-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3514165 INFO (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 3514170 WARN (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3514171 INFO (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 3514176 INFO (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 3514180 INFO (jetty-launcher-6042-thread-2-EventThread) [n:127.0.0.1:57084_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 3514195 WARN (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3514230 INFO (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57084.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e8e7506
[junit4] 2> 3514235 INFO (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57084.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e8e7506
[junit4] 2> 3514235 INFO (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57084.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e8e7506
[junit4] 2> 3514236 INFO (jetty-launcher-6042-thread-2) [n:127.0.0.1:57084_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_74BFBD089602893A-001/tempDir-001/node2/.
[junit4] 2> 3514262 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 3514262 WARN (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3514262 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 3514263 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 3514266 INFO (jetty-launcher-6042-thread-1-EventThread) [n:127.0.0.1:53775_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 3514283 WARN (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 3514310 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53775.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e8e7506
[junit4] 2> 3514315 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53775.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e8e7506
[junit4] 2> 3514315 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53775.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e8e7506
[junit4] 2> 3514316 INFO (jetty-launcher-6042-thread-1) [n:127.0.0.1:53775_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_74BFBD089602893A-001/tempDir-001/node1/.
[junit4] 2> 3514346 INFO (zkConnectionManagerCallback-6067-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3514368 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
[junit4] 2> 3514374 INFO (qtp1056300089-31035) [ ] o.a.s.s.SolrDispatchFilter Error authenticating
[junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:454)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3514374 WARN (qtp1056300089-31035) [ ] o.e.j.s.HttpChannel /solr/admin/collections
[junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:460)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:454)
[junit4] 2> ... 29 more
[junit4] 2> 3514376 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
[junit4] 2> 3514393 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
[junit4] 2> 3514416 WARN (qtp1056300089-31041) [ ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
[junit4] 2> id: invalid user name: "bar"
[junit4] 2>
[junit4] 2> 3514419 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1519248954697&s=kOuAhzm8lflhf0EPrG8ygoJPCIo="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:54 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:54 GMT
[junit4] 2> 3514419 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
[junit4] 2> 3514438 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
[junit4] 2> 3514439 INFO (qtp1056300089-31040) [n:127.0.0.1:57084_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3514439 INFO (qtp1056300089-31040) [n:127.0.0.1:57084_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3514440 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1519248954741&s=LI2/rfrjg3r9wI//lT63YrvbOm8="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:54 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:54 GMT
[junit4] 2> 3514440 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
[junit4] 2> 3514457 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
[junit4] 2> 3514459 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1519248954761&s=c1cEIk2Qk4IvCH6MLM2+xRqpxzc="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:54 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:54 GMT
[junit4] 2> 3514459 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
[junit4] 2> 3514484 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
[junit4] 2> 3514486 INFO (zkConnectionManagerCallback-6070-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3515024 INFO (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3515026 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3515027 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38874/solr ready
[junit4] 2> 3515029 INFO (qtp240362824-31033) [n:127.0.0.1:53775_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3515031 INFO (OverseerThreadFactory-12371-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection forwardingCollection
[junit4] 2> 3515144 INFO (OverseerStateUpdate-72236529632739333-127.0.0.1:53775_solr-n_0000000000) [n:127.0.0.1:53775_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"forwardingCollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"forwardingCollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:53775/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3515348 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3515348 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3515455 INFO (zkCallback-6058-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3515455 INFO (zkCallback-6058-thread-2-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3516368 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 3516383 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
[junit4] 2> 3516462 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 3516484 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
[junit4] 2> 3516485 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53775.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e8e7506
[junit4] 2> 3516486 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3516486 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_74BFBD089602893A-001/tempDir-001/node1/forwardingCollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_74BFBD089602893A-001/tempDir-001/node1/./forwardingCollection_shard1_replica_n1/data/]
[junit4] 2> 3516487 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.22075315483367844]
[junit4] 2> 3516542 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3516542 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3516543 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3516543 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3516544 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=902039819, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 3516544 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4b0119a9[forwardingCollection_shard1_replica_n1] main]
[junit4] 2> 3516546 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 3516549 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 3516566 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3516566 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593010245461868544
[junit4] 2> 3516567 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 3516570 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 3516570 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 3516570 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 3516571 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 3516571 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 3516571 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 3516572 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 3516572 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 3516573 INFO (searcherExecutor-12376-thread-1-processing-n:127.0.0.1:53775_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@4b0119a9[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3516582 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3516582 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3516582 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53775/solr/forwardingCollection_shard1_replica_n1/
[junit4] 2> 3516582 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3516582 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:53775/solr/forwardingCollection_shard1_replica_n1/ has no replicas
[junit4] 2> 3516582 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3516587 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53775/solr/forwardingCollection_shard1_replica_n1/ shard1
[junit4] 2> 3516691 INFO (zkCallback-6058-thread-2-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3516691 INFO (zkCallback-6058-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3516739 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3516743 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1396
[junit4] 2> 3516744 WARN (httpShardExecutor-12366-thread-1-processing-n:127.0.0.1:53775_solr [http:////127.0.0.1:53775//solr] http:////127.0.0.1:53775//solr) [n:127.0.0.1:53775_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1519248955650&s=zVfLb3HD4Vcpfx/CnnNHHzAek4o="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:55 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:55 GMT
[junit4] 2> 3516748 INFO (qtp240362824-31033) [n:127.0.0.1:53775_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3516843 INFO (zkCallback-6058-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3516843 INFO (zkCallback-6058-thread-2-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3517034 INFO (OverseerCollectionConfigSetProcessor-72236529632739333-127.0.0.1:53775_solr-n_0000000000) [n:127.0.0.1:53775_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3517749 INFO (qtp240362824-31033) [n:127.0.0.1:53775_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=2721
[junit4] 2> 3517750 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1519248955331&s=IKPSwPE8akHIE2y1QZ5r1RfQjC4="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:55 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:55 GMT
[junit4] 2> 3517750 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
[junit4] 2> 3517751 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
[junit4] 2> 3517753 INFO (qtp1056300089-31041) [n:127.0.0.1:57084_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3517755 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 3517755 WARN (qtp1056300089-31041) [n:127.0.0.1:57084_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1519248958058&s=z2L9X/ClbueDLWh78oMC7reu6n0="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:58 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:58 GMT
[junit4] 2> 3517756 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1519248958055&s=BTrQ+7d55HSz82maN+Rod33jtnw="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:58 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:58 GMT
[junit4] 2> 3517756 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1519248958058&s=z2L9X/ClbueDLWh78oMC7reu6n0="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:58 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:58 GMT
[junit4] 2> 3517758 INFO (qtp240362824-31046) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 3517758 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1519248958061&s=rPhkG13oRNPSl2xO5PoGFXyOZ7s="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:58 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:58 GMT
[junit4] 2> 3517759 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
[junit4] 2> 3517779 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
[junit4] 2> 3517785 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1519248958084&s=YjNmACea4q7pVyq61HVaTPiXyaM="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:58 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:58 GMT
[junit4] 2> 3517785 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
[junit4] 2> 3517805 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
[junit4] 2> 3517807 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1519248958109&s=5FKLDmiRyA6nIt2mzmkt5HGM+Rw="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:58 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:58 GMT
[junit4] 2> 3517807 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
[junit4] 2> 3517834 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
[junit4] 2> 3517862 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1519248958138&s=jVEoHw3DPcanyRYTc0LmmAb/5FQ="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:58 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:58 GMT
[junit4] 2> 3517862 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
[junit4] 2> 3517888 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
[junit4] 2> 3517906 INFO (qtp1056300089-31041) [n:127.0.0.1:57084_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3517907 INFO (qtp1056300089-31041) [n:127.0.0.1:57084_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3517908 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1519248958192&s=F7ETsk6BfKxnQijaYfxtot8T3Oo="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:58 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:58 GMT
[junit4] 2> 3517908 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
[junit4] 2> 3517930 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
[junit4] 2> 3531786 WARN (jetty-launcher-6042-thread-2-SendThread(127.0.0.1:38874)) [n:127.0.0.1:57084_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 13884ms for sessionid 0x100a2bdb0910006
[junit4] 2> 3531786 WARN (jetty-launcher-6042-thread-1-SendThread(127.0.0.1:38874)) [n:127.0.0.1:53775_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17503ms for sessionid 0x100a2bdb0910008
[junit4] 2> 3531787 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a2bdb0910008, likely client has closed socket
[junit4] 2> 3531787 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a2bdb0910006, likely client has closed socket
[junit4] 2> 3531787 WARN (jetty-launcher-6042-thread-1-SendThread(127.0.0.1:38874)) [n:127.0.0.1:53775_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14757ms for sessionid 0x100a2bdb0910005
[junit4] 2> 3531786 WARN (jetty-launcher-6042-thread-2-SendThread(127.0.0.1:38874)) [n:127.0.0.1:57084_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17504ms for sessionid 0x100a2bdb0910007
[junit4] 2> 3531787 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a2bdb0910007, likely client has closed socket
[junit4] 2> 3531787 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100a2bdb0910005, likely client has closed socket
[junit4] 2> 3531788 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1519248958233&s=smaG+n+ambHQ7px8dHr+STWeL3M="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:35:58 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:35:58 GMT
[junit4] 2> 3531788 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
[junit4] 2> 3531821 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
[junit4] 2> 3531887 WARN (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@123c6b80 name: ZooKeeperConnection Watcher:127.0.0.1:38874/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 3531887 WARN (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 3531887 INFO (jetty-launcher-6042-thread-1-EventThread) [n:127.0.0.1:53775_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 3531890 WARN (zkConnectionManagerCallback-6059-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78639273 name: ZooKeeperConnection Watcher:127.0.0.1:38874/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 3531890 WARN (zkConnectionManagerCallback-6059-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 3531891 WARN (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2048d88a name: ZooKeeperConnection Watcher:127.0.0.1:38874/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 3531891 WARN (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 3546835 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1519248972124&s=tAAaxlFzixDGEwfcL/Q6NEPcKDk="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:36:12 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:36:12 GMT
[junit4] 2> 3546835 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
[junit4] 2> 3546854 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[74BFBD089602893A]-SendThread(127.0.0.1:38874)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100a2bdb091000b has expired
[junit4] 2> 3546861 WARN (jetty-launcher-6042-thread-1-SendThread(127.0.0.1:38874)) [n:127.0.0.1:53775_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100a2bdb0910008 has expired
[junit4] 2> 3546893 WARN (jetty-launcher-6042-thread-2-SendThread(127.0.0.1:38874)) [n:127.0.0.1:57084_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100a2bdb0910007 has expired
[junit4] 2> 3546916 INFO (zkCallback-6058-thread-2-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 3546904 WARN (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2048d88a name: ZooKeeperConnection Watcher:127.0.0.1:38874/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 3546916 WARN (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 3546916 WARN (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 3546890 WARN (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@123c6b80 name: ZooKeeperConnection Watcher:127.0.0.1:38874/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 3546917 WARN (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 3546917 WARN (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 3546904 WARN (jetty-launcher-6042-thread-2-SendThread(127.0.0.1:38874)) [n:127.0.0.1:57084_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100a2bdb0910006 has expired
[junit4] 2> 3546918 INFO (zkConnectionManagerCallback-6073-thread-1-EventThread) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3546896 INFO (zkConnectionManagerCallback-6059-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3546919 INFO (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr-EventThread) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3546919 INFO (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 3546920 INFO (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 3546920 INFO (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 3546896 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
[junit4] 2> 3546922 INFO (qtp1056300089-31040) [n:127.0.0.1:57084_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3546922 INFO (qtp1056300089-31040) [n:127.0.0.1:57084_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3546923 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[74BFBD089602893A]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1519248987225&s=9VjPqq5xU1C+CtEkHUA5LFEwhHI="; Path=/; Domain=127.0.0.1; Expires=mi?, 21-feb-2018 21:36:27 GMT; HttpOnly". Invalid 'expires' attribute: mi?, 21-feb-2018 21:36:27 GMT
[junit4] 2> 3546923 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[74BFBD089602893A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
[junit4] 2> 3546924 INFO (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 3546930 INFO (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57084_solr
[junit4] 2> 3546931 INFO (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
[junit4] 2> 3546931 INFO (zkConnectionManagerCallback-6061-thread-1-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ConnectionManager zkClient Connected:true
[junit4] 2> 3546932 INFO (zkCallback-6058-thread-2-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3546932 INFO (zkCallback-6060-thread-2-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3548425 ERROR (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.ZkStateReader A ZK error has occurred
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1535)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:328)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:673)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$1.command(ZkStateReader.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:167)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:267)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3548426 ERROR (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: A ZK error has occurred
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$1.command(ZkStateReader.java:285)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:167)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:267)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1535)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:328)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:673)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$1.command(ZkStateReader.java:282)
[junit4] 2> ... 10 more
[junit4] 2>
[junit4] 2> 3548426 WARN (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
[junit4] 2> 3548427 INFO (zkConnectionManagerCallback-6073-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient Connected:true
[junit4] 2> 3548446 INFO (jetty-closer-6043-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7e616ef2{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 3548446 INFO (jetty-closer-6043-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@6f5e44f1{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 3548446 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=206319952
[junit4] 2> 3548447 INFO (jetty-closer-6043-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=420321882
[junit4] 2> 3548447 INFO (jetty-closer-6043-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3548447 INFO (jetty-closer-6043-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@b52806f: rootName = solr_57084, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6ecf420b
[junit4] 2> 3548448 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3548448 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4fd16822: rootName = solr_53775, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@340f4c5a
[junit4] 2> 3548452 INFO (jetty-closer-6043-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3548452 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3548452 INFO (jetty-closer-6043-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@110979ef: rootName = solr_57084, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@380c54f7
[junit4] 2> 3548452 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@102b3134: rootName = solr_53775, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@380c54f7
[junit4] 2> 3548455 INFO (jetty-closer-6043-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3548455 INFO (jetty-closer-6043-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3daef6b9: rootName = solr_57084, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@3c33c72e
[junit4] 2> 3548455 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3548455 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@78d71a70: rootName = solr_53775, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@3c33c72e
[junit4] 2> 3548457 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:53775_solr
[junit4] 2> 3548457 INFO (jetty-closer-6043-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:57084_solr
[junit4] 2> 3548458 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3548458 INFO (jetty-closer-6043-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3548459 INFO (zkCallback-6060-thread-2-processing-n:127.0.0.1:57084_solr) [n:127.0.0.1:57084_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 3548459 INFO (coreCloseExecutor-12382-thread-1) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@15f57b22
[junit4] 2> 3548459 INFO (coreCloseExecutor-12382-thread-1) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=368409378
[junit4] 2> 3548459 INFO (coreCloseExecutor-12382-thread-1) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ab4c9b1: rootName = solr_53775, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@3d4197a8
[junit4] 2> 3548459 INFO (zkCallback-6058-thread-2-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 3548461 INFO (zkCallback-6058-thread-7-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [0])
[junit4] 2> 3548461 INFO (zkCallback-6058-thread-1-processing-n:127.0.0.1:53775_solr) [n:127.0.0.1:53775_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [0])
[junit4] 2> 3548463 ERROR (Thread[Thread-6433,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:57084_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 3548465 INFO (jetty-closer-6043-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7cb963{/solr,null,UNAVAILABLE}
[junit4] 2> 3548466 INFO (jetty-closer-6043-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3548469 INFO (coreCloseExecutor-12382-thread-1) [n:127.0.0.1:53775_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=368409378
[junit4] 2> 3548474 INFO (jetty-closer-6043-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72236529632739333-127.0.0.1:53775_solr-n_0000000000) closing
[junit4] 2> 3548475 INFO (OverseerStateUpdate-72236529632739333-127.0.0.1:53775_solr-n_0000000000) [n:127.0.0.1:53775_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:53775_solr
[junit4] 2> 3548475 WARN (OverseerAutoScalingTriggerThread-72236529632739333-127.0.0.1:53775_solr-n_0000000000) [n:127.0.0.1:53775_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3548480 ERROR (Thread[Thread-6436,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:53775_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 3548481 INFO (jetty-closer-6043-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1c983469{/solr,null,UNAVAILABLE}
[junit4] 2> 3548481 INFO (jetty-closer-6043-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3548484 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[74BFBD089602893A]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3548484 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[74BFBD089602893A]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38874 38874
[junit4] 2> 3553624 INFO (Thread-6416) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38874 38874
[junit4] 2> 3553625 WARN (Thread-6416) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 4 /solr/aliases.json
[junit4] 2> 3 /solr/security.json
[junit4] 2> 2 /solr/collections/forwardingCollection/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/forwardingCollection/state.json
[junit4] 2> 4 /solr/clusterstate.json
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
[junit4] 2> 2 /solr/overseer_elect/election/72236529632739333-127.0.0.1:53775_solr-n_0000000000
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/live_nodes
[junit4] 2> 4 /solr/collections
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
[junit4] 2>
[junit4] 2> 3553658 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[74BFBD089602893A]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 21, 2018 11:36:33 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> 3561928 ERROR (jetty-launcher-6042-thread-1-EventThread) [n:127.0.0.1:53775_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
[junit4] 2> java.lang.IllegalStateException: Client is not started
[junit4] 2> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:113)
[junit4] 2> at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:477)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceFacade.getZooKeeper(NamespaceFacade.java:112)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:302)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:291)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[...truncated too long message...]
st, TestNonDefinedSimilarityFactory, TestJsonFacets, CdcrVersionReplicationTest, SegmentsInfoRequestHandlerTest, TestSchemaNameResource, TestImpersonationWithHadoopAuth, BlockCacheTest, MinimalSchemaTest, TestNumericTerms64, TestDistribDocBasedVersion, HttpPartitionOnCommitTest, DocValuesMissingTest, SolrSlf4jReporterTest, TestIntervalFaceting, TestManagedSynonymGraphFilterFactory, TestBlendedInfixSuggestions, CollectionsAPIAsyncDistributedZkTest, TestSQLHandlerNonCloud, BooleanFieldTest, TestLocalFSCloudBackupRestore, TestReplicationHandlerBackup, TestClusterProperties, CloudMLTQParserTest, TestSubQueryTransformer, SolrCoreCheckLockOnStartupTest, TestSolrCoreProperties, OverriddenZkACLAndCredentialsProvidersTest, HdfsCollectionsAPIDistributedZkTest, TestReloadAndDeleteDocs, SuggesterTest, TestWriterPerf, TlogReplayBufferedWhileIndexingTest, LIRRollingUpdatesTest, BasicAuthIntegrationTest, MoveReplicaTest, TestLegacyFieldReuse, TestCrossCoreJoin, DocValuesMultiTest, TestLegacyTerms, TestLMDirichletSimilarityFactory, TestNumericRangeQuery32, TestZkAclsWithHadoopAuth, SimpleCollectionCreateDeleteTest, TestSolrDeletionPolicy2, TestFieldSortValues, NoCacheHeaderTest, HdfsSyncSliceTest, TestAtomicUpdateErrorCases, TestTolerantUpdateProcessorRandomCloud, BasicDistributedZk2Test, TestCollapseQParserPlugin, TestRequestForwarding, TestCollectionsAPIViaSolrCloudCluster, CollectionReloadTest, TestConfig, TestHdfsUpdateLog, ZkSolrClientTest, TestBadConfig, DeleteReplicaTest, TestRTGBase, TestOrdValues, TestFieldCache, TestManagedResourceStorage, HdfsBasicDistributedZk2Test, TestRawTransformer, TestDocSet, CursorMarkTest, TestSimDistributedQueue, HdfsWriteToMultipleCollectionsTest, TestDistribIDF, DistanceFunctionTest, QueryResultKeyTest, ClassificationUpdateProcessorTest, CurrencyFieldTypeTest, TestInitQParser, DistributedExpandComponentTest, JsonLoaderTest, PeerSyncTest, BlockJoinFacetRandomTest, AnalyticsQueryTest, TestSubQueryTransformerDistrib, TestBinaryResponseWriter, SolrCmdDistributorTest, TestDynamicFieldCollectionResource, TestReversedWildcardFilterFactory, TestConfigSetsAPIExclusivity, DistributedQueryComponentOptimizationTest, SpatialHeatmapFacetsTest, TestExportWriter, TestGraphTermsQParserPlugin, TestPKIAuthenticationPlugin, HdfsChaosMonkeyNothingIsSafeTest, ClassificationUpdateProcessorFactoryTest, TestLeaderInitiatedRecoveryThread, RuleEngineTest, SolrJmxReporterCloudTest, RemoteQueryErrorTest, TestCorePropertiesReload, AutoscalingHistoryHandlerTest, TestValueSourceCache, TestReRankQParserPlugin, PrimitiveFieldTypeTest, PeerSyncWithIndexFingerprintCachingTest, TestRTimerTree, DocValuesTest, CleanupOldIndexTest, ResponseHeaderTest, TestDirectoryFactory, SimplePostToolTest, BasicDistributedZkTest, UnloadDistributedZkTest, SyncSliceTest, OverseerTest, LeaderElectionIntegrationTest, TestRandomFaceting, TestZkChroot, TestRandomDVFaceting, TestDistributedSearch, TestRealTimeGet, TestJoin, TestRangeQuery, SimpleFacetsTest, SolrCoreTest, SpellCheckComponentTest, QueryElevationComponentTest, BadIndexSchemaTest, ConvertedLegacyTest, BasicFunctionalityTest, TestLazyCores, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SpellCheckCollatorTest, TestFoldingMultitermQuery, SpatialFilterTest, SuggesterWFSTTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestUpdate, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, TestRemoteStreaming, XsltUpdateRequestHandlerTest, TestSurroundQueryParser, LukeRequestHandlerTest, TestQueryUtils, DocumentAnalysisRequestHandlerTest, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, MoreLikeThisHandlerTest, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, IndexSchemaTest, TestJmxIntegration, ReturnFieldsTest, MBeansHandlerTest, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, PingRequestHandlerTest, TestComponentsName, SearchHandlerTest, TestLFUCache, SolrIndexConfigTest, TestSearchPerf, NumericFieldsTest, TestFuzzyAnalyzedSuggestions, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TestFastOutputStream, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, PluginInfoTest, TestFastLRUCache, DateMathParserTest, ChaosMonkeyNothingIsSafeTest, AssignBackwardCompatibilityTest, ChaosMonkeySafeLeaderWithPullReplicasTest, ConfigSetsAPITest, CreateRoutedAliasTest, DeleteLastCustomShardedReplicaTest, DeleteStatusTest, DistribDocExpirationUpdateProcessorTest, HttpPartitionTest, LeaderFailoverAfterPartitionTest, MoveReplicaHDFSFailoverTest, RestartWhileUpdatingTest, RollingRestartTest, SolrCloudExampleTest, TestCloudPseudoReturnFields, TestCloudRecovery, TestCryptoKeys, TestLockTree, TestMiniSolrCloudClusterSSL, TestPullReplicaErrorHandling, TestSegmentSorting, TestSizeLimitedDistributedMap, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=74BFBD089602893A -Dtests.slow=true -Dtests.locale=es-US -Dtests.timezone=BET -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=31168, name=jetty-launcher-6042-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4] > 2) Thread[id=31083, name=jetty-launcher-6042-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] > 3) Thread[id=31167, name=jetty-launcher-6042-thread-1-SendThread(127.0.0.1:38874), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > at __randomizedtesting.SeedInfo.seed([74BFBD089602893A]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=31167, name=jetty-launcher-6042-thread-1-SendThread(127.0.0.1:38874), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > at __randomizedtesting.SeedInfo.seed([74BFBD089602893A]:0)
[junit4] Completed [581/777 (1!)] on J1 in 62.95s, 12 tests, 2 errors <<< FAILURES!
[...truncated 49123 lines...]
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 454 -
Still unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/454/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=19728, name=jetty-launcher-3961-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) 2) Thread[id=19809, name=jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 3) Thread[id=19810, name=jetty-launcher-3961-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=19728, name=jetty-launcher-3961-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
2) Thread[id=19809, name=jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
3) Thread[id=19810, name=jetty-launcher-3961-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
at __randomizedtesting.SeedInfo.seed([C784ACD9BAE1F47F]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=19809, name=jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=19809, name=jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
at __randomizedtesting.SeedInfo.seed([C784ACD9BAE1F47F]:0)
Build Log:
[...truncated 13210 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_C784ACD9BAE1F47F-001/init-core-data-001
[junit4] 2> 2909220 WARN (SUITE-TestSolrCloudWithSecureImpersonation-seed#[C784ACD9BAE1F47F]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=39 numCloses=39
[junit4] 2> 2909221 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[C784ACD9BAE1F47F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2909222 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[C784ACD9BAE1F47F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2909222 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[C784ACD9BAE1F47F]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2909249 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[C784ACD9BAE1F47F]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_C784ACD9BAE1F47F-001/tempDir-001
[junit4] 2> 2909249 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[C784ACD9BAE1F47F]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2909249 INFO (Thread-6080) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2909249 INFO (Thread-6080) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2909251 ERROR (Thread-6080) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2909349 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[C784ACD9BAE1F47F]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:58653
[junit4] 2> 2909352 INFO (zkConnectionManagerCallback-3964-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2909356 INFO (jetty-launcher-3961-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2909356 INFO (jetty-launcher-3961-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2909357 INFO (jetty-launcher-3961-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2909357 INFO (jetty-launcher-3961-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2909357 INFO (jetty-launcher-3961-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2909358 INFO (jetty-launcher-3961-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2909358 INFO (jetty-launcher-3961-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2909358 INFO (jetty-launcher-3961-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2909358 INFO (jetty-launcher-3961-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b883ba6{/solr,null,AVAILABLE}
[junit4] 2> 2909359 INFO (jetty-launcher-3961-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@18d8f4d6{/solr,null,AVAILABLE}
[junit4] 2> 2909360 INFO (jetty-launcher-3961-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@41119519{HTTP/1.1,[http/1.1]}{127.0.0.1:52676}
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-2) [ ] o.e.j.s.Server Started @2915752ms
[junit4] 2> 2909360 INFO (jetty-launcher-3961-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1c4e5c09{HTTP/1.1,[http/1.1]}{127.0.0.1:53763}
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52676}
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-1) [ ] o.e.j.s.Server Started @2915753ms
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53763}
[junit4] 2> 2909361 ERROR (jetty-launcher-3961-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2909361 ERROR (jetty-launcher-3961-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-20T23:49:52.099Z
[junit4] 2> 2909361 INFO (jetty-launcher-3961-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-20T23:49:52.099Z
[junit4] 2> 2909363 INFO (zkConnectionManagerCallback-3968-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2909363 INFO (zkConnectionManagerCallback-3966-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2909364 INFO (jetty-launcher-3961-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2909364 INFO (jetty-launcher-3961-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2909371 INFO (jetty-launcher-3961-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58653/solr
[junit4] 2> 2909371 INFO (jetty-launcher-3961-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58653/solr
[junit4] 2> 2909373 INFO (zkConnectionManagerCallback-3974-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2909373 INFO (zkConnectionManagerCallback-3976-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2909376 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2909376 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2909507 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2909508 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52676_solr
[junit4] 2> 2909509 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.c.Overseer Overseer (id=72233749789474821-127.0.0.1:52676_solr-n_0000000000) starting
[junit4] 2> 2909512 INFO (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2909513 INFO (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53763_solr
[junit4] 2> 2909515 INFO (zkCallback-3977-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2909516 INFO (zkCallback-3979-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2909522 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52676_solr
[junit4] 2> 2909523 INFO (zkCallback-3977-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2909524 INFO (zkCallback-3979-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2909556 INFO (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 2909556 WARN (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 2909556 INFO (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 2909558 INFO (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 2909560 INFO (jetty-launcher-3961-thread-1-EventThread) [n:127.0.0.1:53763_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 2909574 WARN (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 2909599 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 2909599 WARN (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 2909599 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 2909600 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 2909602 INFO (jetty-launcher-3961-thread-2-EventThread) [n:127.0.0.1:52676_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 2909612 INFO (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53763.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@96385d5
[junit4] 2> 2909614 WARN (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 2909619 INFO (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53763.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@96385d5
[junit4] 2> 2909619 INFO (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53763.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@96385d5
[junit4] 2> 2909621 INFO (jetty-launcher-3961-thread-1) [n:127.0.0.1:53763_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_C784ACD9BAE1F47F-001/tempDir-001/node2/.
[junit4] 2> 2909648 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52676.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@96385d5
[junit4] 2> 2909654 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52676.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@96385d5
[junit4] 2> 2909654 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52676.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@96385d5
[junit4] 2> 2909669 INFO (jetty-launcher-3961-thread-2) [n:127.0.0.1:52676_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_C784ACD9BAE1F47F-001/tempDir-001/node1/.
[junit4] 2> 2909685 INFO (zkConnectionManagerCallback-3986-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2909706 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
[junit4] 2> 2935137 WARN (jetty-launcher-3961-thread-1-SendThread(127.0.0.1:58653)) [n:127.0.0.1:53763_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 25519ms for sessionid 0x100a03674ff0007
[junit4] 2> 2935138 WARN (jetty-launcher-3961-thread-1-SendThread(127.0.0.1:58653)) [n:127.0.0.1:53763_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 25525ms for sessionid 0x100a03674ff0006
[junit4] 2> 2935138 WARN (jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653)) [n:127.0.0.1:52676_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 25477ms for sessionid 0x100a03674ff0005
[junit4] 2> 2935139 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1519206592447&s=mTKWLG5yHBV4sPWF94Z6Pkh7uxI="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:49:52 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:49:52 GMT
[junit4] 2> 2935140 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
[junit4] 2> 2935160 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
[junit4] 2> 2935238 INFO (jetty-launcher-3961-thread-2-EventThread) [n:127.0.0.1:52676_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 2935240 WARN (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@35d35bd1 name: ZooKeeperConnection Watcher:127.0.0.1:58653/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 2935241 WARN (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 2935242 WARN (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@ee23c9e name: ZooKeeperConnection Watcher:127.0.0.1:58653/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 2935242 WARN (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 2957650 WARN (jetty-launcher-3961-thread-1-SendThread(127.0.0.1:58653)) [n:127.0.0.1:53763_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 22420ms for sessionid 0x100a03674ff0007
[junit4] 2> 2957650 WARN (jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653)) [n:127.0.0.1:52676_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 22418ms for sessionid 0x100a03674ff0005
[junit4] 2> 2957651 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x0, likely client has closed socket
[junit4] 2> 2957652 WARN (jetty-launcher-3961-thread-1-SendThread(127.0.0.1:58653)) [n:127.0.0.1:53763_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100a03674ff0006 has expired
[junit4] 2> 2957652 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x0, likely client has closed socket
[junit4] 2> 2957652 WARN (jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653)) [n:127.0.0.1:52676_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100a03674ff0008 has expired
[junit4] 2> 2957653 WARN (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@ee23c9e name: ZooKeeperConnection Watcher:127.0.0.1:58653/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 2957653 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1519206617899&s=hvJGV3BNZRE74d+5Cqh+Mgeh0d0="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:17 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:17 GMT
[junit4] 2> 2957653 WARN (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 2957653 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2957653 WARN (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 2957653 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
[junit4] 2> 2957657 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr-EventThread) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2957658 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 2957658 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 2957664 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 2957681 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
[junit4] 2> 2957682 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53763_solr
[junit4] 2> 2957683 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1519206640420&s=0xY8k4FyCJr9OVuvBLjbksEYrY4="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:40 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:40 GMT
[junit4] 2> 2957684 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
[junit4] 2> 2957684 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.Overseer Overseer (id=72233749789474826-127.0.0.1:53763_solr-n_0000000002) starting
[junit4] 2> 2957711 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
[junit4] 2> 2957723 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53763_solr
[junit4] 2> 2957724 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
[junit4] 2> 2957724 INFO (zkConnectionManagerCallback-3978-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ConnectionManager zkClient Connected:true
[junit4] 2> 2957726 INFO (OverseerStateUpdate-72233749789474826-127.0.0.1:53763_solr-n_0000000002) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2957733 WARN (qtp1388351629-19689) [ ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
[junit4] 2> id: invalid user name: "bar"
[junit4] 2>
[junit4] 2> 2957734 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1519206640449&s=Vtn+XyKvzBkwCiYVGnS7eRi2KhI="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:40 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:40 GMT
[junit4] 2> 2957735 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
[junit4] 2> 2957765 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
[junit4] 2> 2958165 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1519206640503&s=S03BinzQ64iefEnPWHcBjmnszy0="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:40 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:40 GMT
[junit4] 2> 2958165 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
[junit4] 2> 2958195 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
[junit4] 2> 2958211 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2958211 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2958212 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1519206640933&s=BkKxQBd7Xl8/s/sgtUf8kusssCs="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:40 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:40 GMT
[junit4] 2> 2958212 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
[junit4] 2> 2958235 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
[junit4] 2> 2958237 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1519206640973&s=a0F0kGRLndVfnCuEWte44pqKiWE="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:40 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:40 GMT
[junit4] 2> 2958237 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
[junit4] 2> 2958273 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
[junit4] 2> 2958273 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2958273 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2958274 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1519206641011&s=dcxnMvu0tTJqfW5sHQ9CzNyZdos="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:41 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:41 GMT
[junit4] 2> 2958274 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
[junit4] 2> 2958297 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
[junit4] 2> 2958297 INFO (qtp1388351629-19690) [ ] o.a.s.s.SolrDispatchFilter Error authenticating
[junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:454)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2958298 WARN (qtp1388351629-19690) [ ] o.e.j.s.HttpChannel /solr/admin/collections
[junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:460)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:454)
[junit4] 2> ... 31 more
[junit4] 2> 2958304 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
[junit4] 2> 2958337 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
[junit4] 2> 2958338 INFO (zkConnectionManagerCallback-3989-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2958853 INFO (zkConnectionManagerCallback-3992-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2958855 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2958856 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58653/solr ready
[junit4] 2> 2958857 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2958860 INFO (OverseerThreadFactory-7206-thread-1-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection forwardingCollection
[junit4] 2> 2958934 WARN (jetty-launcher-3961-thread-1-SendThread(127.0.0.1:58653)) [n:127.0.0.1:53763_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100a03674ff0007 has expired
[junit4] 2> 2958972 INFO (OverseerStateUpdate-72233749789474826-127.0.0.1:53763_solr-n_0000000002) [n:127.0.0.1:53763_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"forwardingCollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"forwardingCollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:53763/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2958974 WARN (jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653)) [n:127.0.0.1:52676_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100a03674ff0005 has expired
[junit4] 2> 2958974 WARN (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@35d35bd1 name: ZooKeeperConnection Watcher:127.0.0.1:58653/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 2958974 WARN (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 2958975 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.Overseer Overseer (id=72233749789474821-127.0.0.1:52676_solr-n_0000000000) closing
[junit4] 2> 2958975 INFO (OverseerCollectionConfigSetProcessor-72233749789474821-127.0.0.1:52676_solr-n_0000000000) [n:127.0.0.1:52676_solr ] o.a.s.c.OverseerTaskProcessor According to ZK I (id=72233749789474821-127.0.0.1:52676_solr-n_0000000000) am no longer a leader.
[junit4] 2> 2958975 INFO (OverseerStateUpdate-72233749789474821-127.0.0.1:52676_solr-n_0000000000) [n:127.0.0.1:52676_solr ] o.a.s.c.Overseer According to ZK I (id=72233749789474821-127.0.0.1:52676_solr-n_0000000000) am no longer a leader.
[junit4] 2> 2958975 INFO (OverseerStateUpdate-72233749789474821-127.0.0.1:52676_solr-n_0000000000) [n:127.0.0.1:52676_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:52676_solr
[junit4] 2> 2958976 WARN (OverseerAutoScalingTriggerThread-72233749789474821-127.0.0.1:52676_solr-n_0000000000) [n:127.0.0.1:52676_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2958976 ERROR (OverseerExitThread) [n:127.0.0.1:52676_solr ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:306)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2958976 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2958976 WARN (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 2958978 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr-EventThread) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2958978 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 2958978 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 2958980 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 2958987 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52676_solr
[junit4] 2> 2958988 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
[junit4] 2> 2958988 INFO (zkConnectionManagerCallback-3980-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ConnectionManager zkClient Connected:true
[junit4] 2> 2958988 INFO (zkCallback-3977-thread-2-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2958988 INFO (zkCallback-3979-thread-3-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2958988 ERROR (OverseerStateUpdate-72233749789474826-127.0.0.1:53763_solr-n_0000000002) [n:127.0.0.1:53763_solr ] o.a.s.c.Overseer Overseer could not process the current clusterstate state update message, skipping the message: {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:52676_solr"}
[junit4] 2> java.lang.RuntimeException: Replica without node name! core_node2:{"core":"forwardingCollection_shard1_replica_n1","base_url":"http://127.0.0.1:53763/solr","state":"down","node_name":null,"type":"NRT"}
[junit4] 2> at org.apache.solr.cloud.overseer.NodeMutator.downNode(NodeMutator.java:62)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:402)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processQueueItem(Overseer.java:278)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:226)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2958989 INFO (zkCallback-3991-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2959177 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2959283 INFO (zkCallback-3977-thread-2-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2959283 INFO (zkCallback-3977-thread-6-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2960197 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2960216 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
[junit4] 2> 2960296 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2960315 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
[junit4] 2> 2960316 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53763.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@96385d5
[junit4] 2> 2960317 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2960317 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_C784ACD9BAE1F47F-001/tempDir-001/node2/forwardingCollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_C784ACD9BAE1F47F-001/tempDir-001/node2/./forwardingCollection_shard1_replica_n1/data/]
[junit4] 2> 2960318 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=35.5751953125, floorSegmentMB=2.189453125, forceMergeDeletesPctAllowed=0.5848840689025203, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2984971268336978
[junit4] 2> 2960372 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2960372 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2960374 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2960374 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2960375 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1064513184, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2960375 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6556a9a1[forwardingCollection_shard1_replica_n1] main]
[junit4] 2> 2960377 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 2960381 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 2960396 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2960396 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 2960396 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592965876294877184
[junit4] 2> 2960397 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 2960397 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 2960397 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 2960397 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 2960397 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 2960398 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 2960398 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 2960399 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 2960401 INFO (searcherExecutor-7207-thread-1-processing-n:127.0.0.1:53763_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@6556a9a1[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2960406 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2960406 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2960406 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53763/solr/forwardingCollection_shard1_replica_n1/
[junit4] 2> 2960406 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2960406 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:53763/solr/forwardingCollection_shard1_replica_n1/ has no replicas
[junit4] 2> 2960406 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2960410 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53763/solr/forwardingCollection_shard1_replica_n1/ shard1
[junit4] 2> 2960512 INFO (zkCallback-3977-thread-6-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2960512 INFO (zkCallback-3977-thread-2-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2960565 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2960567 INFO (qtp1388351629-19687) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1390
[junit4] 2> 2960568 WARN (httpShardExecutor-7193-thread-1-processing-n:127.0.0.1:53763_solr [http:////127.0.0.1:53763//solr] http:////127.0.0.1:53763//solr) [n:127.0.0.1:53763_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1519206641915&s=LuqpKJomELiXR+2ghWJHcPe3Qek="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:41 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:41 GMT
[junit4] 2> 2960570 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2960669 INFO (zkCallback-3977-thread-6-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2960669 INFO (zkCallback-3977-thread-2-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2960862 INFO (OverseerCollectionConfigSetProcessor-72233749789474826-127.0.0.1:53763_solr-n_0000000002) [n:127.0.0.1:53763_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2961571 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=2714
[junit4] 2> 2961572 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1519206641595&s=kjxsEkbS8s56VVWnxnWVLwmXhxM="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:41 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:41 GMT
[junit4] 2> 2961572 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
[junit4] 2> 2961572 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
[junit4] 2> 2961575 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 2961575 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1519206644312&s=OoK2rfBEFKb3/xbrRnzZzUY1qHw="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:44 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:44 GMT
[junit4] 2> 2961581 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 2961581 WARN (qtp1638741252-19679) [n:127.0.0.1:52676_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1519206644319&s=FzX2Ag4+7P4Q0ygLOsxjeU+5rYs="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:44 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:44 GMT
[junit4] 2> 2961582 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1519206644317&s=3P9WAvq/oNvCnT9906ghcoUu6dU="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:44 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:44 GMT
[junit4] 2> 2961582 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1519206644319&s=FzX2Ag4+7P4Q0ygLOsxjeU+5rYs="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:44 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:44 GMT
[junit4] 2> 2961582 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
[junit4] 2> 2961616 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
[junit4] 2> 2961617 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2961618 INFO (qtp1388351629-19689) [n:127.0.0.1:53763_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2961618 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1519206644355&s=V/TuYmFxkPuUbYGxbvA4o7yWj7M="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:44 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:44 GMT
[junit4] 2> 2961618 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
[junit4] 2> 2961652 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
[junit4] 2> 2961655 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[C784ACD9BAE1F47F]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1519206644391&s=OyReBkx+fDskrkb46MvNkvO24LQ="; Path=/; Domain=127.0.0.1; Expires=Rab, 21-Feb-2018 09:50:44 GMT; HttpOnly". Invalid 'expires' attribute: Rab, 21-Feb-2018 09:50:44 GMT
[junit4] 2> 2961655 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[C784ACD9BAE1F47F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
[junit4] 2> 2961657 INFO (jetty-closer-3962-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1c4e5c09{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2961658 INFO (jetty-closer-3962-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@41119519{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2961659 INFO (jetty-closer-3962-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1826837385
[junit4] 2> 2961659 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=821383207
[junit4] 2> 2961659 INFO (jetty-closer-3962-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2961659 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2961659 INFO (jetty-closer-3962-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@fec3b4b: rootName = solr_52676, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@66b634a3
[junit4] 2> 2961660 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@266a7258: rootName = solr_53763, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@53bef8d5
[junit4] 2> 2961670 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2961670 INFO (jetty-closer-3962-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2961670 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@495e36a0: rootName = solr_53763, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3a456906
[junit4] 2> 2961670 INFO (jetty-closer-3962-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3467b037: rootName = solr_52676, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3a456906
[junit4] 2> 2961675 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2961675 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@781b75ff: rootName = solr_53763, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1115636a
[junit4] 2> 2961676 INFO (jetty-closer-3962-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2961676 INFO (jetty-closer-3962-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@34474e28: rootName = solr_52676, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1115636a
[junit4] 2> 2961677 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:53763_solr
[junit4] 2> 2961678 INFO (jetty-closer-3962-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:52676_solr
[junit4] 2> 2961678 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 2961679 INFO (zkCallback-3977-thread-2-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 2961679 INFO (coreCloseExecutor-7212-thread-1) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@702b7b0d
[junit4] 2> 2961679 INFO (coreCloseExecutor-7212-thread-1) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=1881897741
[junit4] 2> 2961679 INFO (coreCloseExecutor-7212-thread-1) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7369bf3: rootName = solr_53763, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@7be63783
[junit4] 2> 2961679 INFO (zkCallback-3979-thread-3-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 2961680 INFO (jetty-closer-3962-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 2961682 WARN (zkCallback-3979-thread-3-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.ZkController Unable to register nodeLost path for 127.0.0.1:53763_solr
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeLost/127.0.0.1:53763_solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:885)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:704)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:703)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1090)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1085)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:269)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2961686 INFO (zkCallback-3977-thread-6-processing-n:127.0.0.1:53763_solr) [n:127.0.0.1:53763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 2961687 WARN (zkCallback-3979-thread-1-processing-n:127.0.0.1:52676_solr) [n:127.0.0.1:52676_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2961689 ERROR (Thread[Thread-6099,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:52676_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 2961692 INFO (jetty-closer-3962-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5b883ba6{/solr,null,UNAVAILABLE}
[junit4] 2> 2961693 INFO (jetty-closer-3962-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2961701 INFO (coreCloseExecutor-7212-thread-1) [n:127.0.0.1:53763_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=1881897741
[junit4] 2> 2961703 INFO (jetty-closer-3962-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72233749789474826-127.0.0.1:53763_solr-n_0000000002) closing
[junit4] 2> 2961703 INFO (OverseerStateUpdate-72233749789474826-127.0.0.1:53763_solr-n_0000000002) [n:127.0.0.1:53763_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:53763_solr
[junit4] 2> 2961704 WARN (OverseerAutoScalingTriggerThread-72233749789474826-127.0.0.1:53763_solr-n_0000000002) [n:127.0.0.1:53763_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2961709 ERROR (Thread[Thread-6097,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:53763_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 2961712 INFO (jetty-closer-3962-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@18d8f4d6{/solr,null,UNAVAILABLE}
[junit4] 2> 2961714 INFO (jetty-closer-3962-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2961715 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[C784ACD9BAE1F47F]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2961716 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[C784ACD9BAE1F47F]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58653 58653
[junit4] 2> 2978152 ERROR (jetty-launcher-3961-thread-2-EventThread) [n:127.0.0.1:52676_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
[junit4] 2> java.lang.IllegalStateException: Client is not started
[junit4] 2> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:113)
[junit4] 2> at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:477)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceFacade.getZooKeeper(NamespaceFacade.java:112)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:302)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:291)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] 2> at or
[...truncated too long message...]
eImpersonation]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1846, maxMBSortInHeap=7.11965785245542, sim=RandomSimilarity(queryNorm=false): {}, locale=id-ID, timezone=America/Jamaica
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=4,free=162023120,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestNestedDocsSort, TestSolrDeletionPolicy2, TestSimDistributedQueue, SolrShardReporterTest, TriggerIntegrationTest, TestValueSourceCache, SolrJmxReporterCloudTest, QueryResultKeyTest, TestDocSet, SparseHLLTest, SegmentsInfoRequestHandlerTest, DisMaxRequestHandlerTest, TestManagedSchema, CSVRequestHandlerTest, DocValuesTest, SimpleFacetsTest, HighlighterConfigTest, SubstringBytesRefFilterTest, UpdateParamsTest, TestDocumentBuilder, PeerSyncWithIndexFingerprintCachingTest, TestManagedResourceStorage, TestUpdate, TestFieldCache, TestHdfsUpdateLog, XsltUpdateRequestHandlerTest, HdfsBasicDistributedZk2Test, NodeMutatorTest, HdfsCollectionsAPIDistributedZkTest, AutoAddReplicasPlanActionTest, TestNumericTokenStream, TestMaxTokenLenTokenizer, SolrGangliaReporterTest, TestManagedStopFilterFactory, TestNumericTerms64, TestFieldCollectionResource, TestMultiWordSynonyms, TestTestInjection, DocValuesNotIndexedTest, CollectionStateFormat2Test, TestMaxScoreQueryParser, UninvertDocValuesMergePolicyTest, CachingDirectoryFactoryTest, ConnectionManagerTest, SSLMigrationTest, HdfsLockFactoryTest, QueryParsingTest, HdfsSyncSliceTest, NotRequiredUniqueKeyTest, JSONWriterTest, TestUnifiedSolrHighlighter, HdfsChaosMonkeyNothingIsSafeTest, AddBlockUpdateTest, TestCollectionsAPIViaSolrCloudCluster, TestManagedSchemaThreadSafety, NodeLostTriggerTest, TestStressUserVersions, SystemInfoHandlerTest, TestPayloadScoreQParserPlugin, ExplicitHLLTest, BlockCacheTest, TestConfig, DocValuesMultiTest, ClusterStateUpdateTest, TestLeaderElectionWithEmptyReplica, TestSubQueryTransformerDistrib, MoveReplicaTest, TestMiniSolrCloudClusterSSL, SpellCheckCollatorTest, TestRandomDVFaceting, AnalyticsQueryTest, TestFieldCacheVsDocValues, ScriptEngineTest, TestDistributedSearch, SuggestComponentTest, LIRRollingUpdatesTest, DistributedExpandComponentTest, FileBasedSpellCheckerTest, TestSortableTextField, SuggesterWFSTTest, TestZkAclsWithHadoopAuth, TestCorePropertiesReload, TestSimpleQParserPlugin, BadComponentTest, TestManagedSynonymGraphFilterFactory, TestCryptoKeys, DeleteStatusTest, TestReversedWildcardFilterFactory, UpdateRequestProcessorFactoryTest, TestSolrQueryResponse, ChaosMonkeySafeLeaderTest, AsyncCallRequestStatusResponseTest, SolrCoreCheckLockOnStartupTest, BasicDistributedZk2Test, SpellCheckCollatorWithCollapseTest, SolrMetricReporterTest, TestSegmentSorting, VMParamsZkACLAndCredentialsProvidersTest, JsonLoaderTest, TestWordDelimiterFilterFactory, TestDirectoryFactory, AtomicUpdateProcessorFactoryTest, TestSerializedLuceneMatchVersion, HLLSerializationTest, DebugComponentTest, TestSolrXml, ZkStateWriterTest, TestRequestStatusCollectionAPI, TestSolrConfigHandlerConcurrent, TestLMJelinekMercerSimilarityFactory, TestCrossCoreJoin, TestDynamicFieldCollectionResource, TestLegacyFieldCache, CursorMarkTest, LeaderFailoverAfterPartitionTest, DeleteLastCustomShardedReplicaTest, ResponseLogComponentTest, CdcrBootstrapTest, DistanceFunctionTest, TestCoreBackup, JavabinLoaderTest, ZkSolrClientTest, TestBlendedInfixSuggestions, TestPKIAuthenticationPlugin, TestComponentsName, RegexBoostProcessorTest, TestReloadAndDeleteDocs, TestConfigSetProperties, DefaultValueUpdateProcessorTest, TestStressLucene, DirectUpdateHandlerTest, BJQParserTest, DOMUtilTest, PKIAuthenticationIntegrationTest, TestLuceneIndexBackCompat, CreateRoutedAliasTest, TestRestoreCore, ExternalFileFieldSortTest, LoggingHandlerTest, TestSchemaManager, TestRTimerTree, TestQuerySenderListener, CoreSorterTest, CheckHdfsIndexTest, SpatialHeatmapFacetsTest, SystemLogListenerTest, TestNumericRangeQuery32, TestFieldCacheSort, TestTlogReplica, PeerSyncReplicationTest, TestPullReplicaErrorHandling, CoreAdminHandlerTest, TestJsonFacets, V2ApiIntegrationTest, PreAnalyzedFieldTest, ClusterStateTest, TestInitParams, TestComplexPhraseQParserPlugin, TestLMDirichletSimilarityFactory, TestSolrCloudWithDelegationTokens, TestFieldTypeCollectionResource, ResponseBuilderTest, TestLuceneMatchVersion, TestSearchPerf, TestDFRSimilarityFactory, OverseerModifyCollectionTest, TestSchemaNameResource, MetricTriggerTest, TestAnalyzedSuggestions, TestExtendedDismaxParser, SpellCheckComponentTest, LeaderInitiatedRecoveryOnShardRestartTest, TestSubQueryTransformer, TolerantUpdateProcessorTest, HdfsTlogReplayBufferedWhileIndexingTest, NoCacheHeaderTest, TestConfigSetsAPIExclusivity, TestInitQParser, TestDistributedMissingSort, PrimitiveFieldTypeTest, TestApiFramework, TestCloudPivotFacet, BlobRepositoryCloudTest, DistributedTermsComponentTest, SyncSliceTest, TestSolrCoreSnapshots, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=C784ACD9BAE1F47F -Dtests.slow=true -Dtests.locale=id-ID -Dtests.timezone=America/Jamaica -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=19728, name=jetty-launcher-3961-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] > 2) Thread[id=19809, name=jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > 3) Thread[id=19810, name=jetty-launcher-3961-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4] > at __randomizedtesting.SeedInfo.seed([C784ACD9BAE1F47F]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=19809, name=jetty-launcher-3961-thread-2-SendThread(127.0.0.1:58653), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > at __randomizedtesting.SeedInfo.seed([C784ACD9BAE1F47F]:0)
[junit4] Completed [340/777 (1!)] on J1 in 92.21s, 12 tests, 2 errors <<< FAILURES!
[...truncated 49892 lines...]
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 453 -
Failure!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/453/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC
4 tests failed.
FAILED: org.apache.solr.cloud.AliasIntegrationTest.testMetadata
Error Message:
Error from server at http://127.0.0.1:52458/solr: Collection : collection2meta is part of alias meta3 remove or modify the alias before removing this collection.
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:52458/solr: Collection : collection2meta is part of alias meta3 remove or modify the alias before removing this collection.
at __randomizedtesting.SeedInfo.seed([388583CDDAA1B4E0:3195C9C99D48E889]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:444)
at org.apache.solr.cloud.AliasIntegrationTest.tearDown(AliasIntegrationTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ReplaceNodeNoTargetTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([388583CDDAA1B4E0:B0D1BC17745DD918]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=14459, name=jetty-launcher-2878-thread-2-SendThread(127.0.0.1:56087), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 2) Thread[id=14460, name=jetty-launcher-2878-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 3) Thread[id=14392, name=jetty-launcher-2878-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=14459, name=jetty-launcher-2878-thread-2-SendThread(127.0.0.1:56087), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
2) Thread[id=14460, name=jetty-launcher-2878-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
3) Thread[id=14392, name=jetty-launcher-2878-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
at __randomizedtesting.SeedInfo.seed([388583CDDAA1B4E0]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=14459, name=jetty-launcher-2878-thread-2-SendThread(127.0.0.1:56087), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=14459, name=jetty-launcher-2878-thread-2-SendThread(127.0.0.1:56087), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
at __randomizedtesting.SeedInfo.seed([388583CDDAA1B4E0]:0)
Build Log:
[...truncated 12319 lines...]
[junit4] Suite: org.apache.solr.cloud.AliasIntegrationTest
[junit4] 2> 439557 INFO (SUITE-AliasIntegrationTest-seed#[388583CDDAA1B4E0]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/init-core-data-001
[junit4] 2> 439558 INFO (SUITE-AliasIntegrationTest-seed#[388583CDDAA1B4E0]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 439558 INFO (SUITE-AliasIntegrationTest-seed#[388583CDDAA1B4E0]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 439559 INFO (SUITE-AliasIntegrationTest-seed#[388583CDDAA1B4E0]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/tempDir-001
[junit4] 2> 439559 INFO (SUITE-AliasIntegrationTest-seed#[388583CDDAA1B4E0]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 439559 INFO (Thread-988) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 439559 INFO (Thread-988) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 439561 ERROR (Thread-988) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 439659 INFO (SUITE-AliasIntegrationTest-seed#[388583CDDAA1B4E0]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:38847
[junit4] 2> 439661 INFO (zkConnectionManagerCallback-834-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 439665 INFO (jetty-launcher-831-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 439665 INFO (jetty-launcher-831-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 439667 INFO (jetty-launcher-831-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 439667 INFO (jetty-launcher-831-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 439667 INFO (jetty-launcher-831-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 439667 INFO (jetty-launcher-831-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@f5efcca{/solr,null,AVAILABLE}
[junit4] 2> 439667 INFO (jetty-launcher-831-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7ea42264{HTTP/1.1,[http/1.1]}{127.0.0.1:34724}
[junit4] 2> 439667 INFO (jetty-launcher-831-thread-2) [ ] o.e.j.s.Server Started @442746ms
[junit4] 2> 439667 INFO (jetty-launcher-831-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34724}
[junit4] 2> 439667 ERROR (jetty-launcher-831-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 439668 INFO (jetty-launcher-831-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 439668 INFO (jetty-launcher-831-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 439668 INFO (jetty-launcher-831-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 439668 INFO (jetty-launcher-831-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 439668 INFO (jetty-launcher-831-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@9b9e88{/solr,null,AVAILABLE}
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-20T13:57:48.016Z
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@61706f8e{HTTP/1.1,[http/1.1]}{127.0.0.1:52458}
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-1) [ ] o.e.j.s.Server Started @442747ms
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52458}
[junit4] 2> 439669 ERROR (jetty-launcher-831-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 439669 INFO (jetty-launcher-831-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-20T13:57:48.016Z
[junit4] 2> 439672 INFO (zkConnectionManagerCallback-838-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 439672 INFO (zkConnectionManagerCallback-836-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 439672 INFO (jetty-launcher-831-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 439672 INFO (jetty-launcher-831-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 439685 INFO (jetty-launcher-831-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38847/solr
[junit4] 2> 439687 INFO (jetty-launcher-831-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38847/solr
[junit4] 2> 439693 INFO (zkConnectionManagerCallback-842-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 439694 INFO (zkConnectionManagerCallback-846-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 439703 INFO (zkConnectionManagerCallback-848-thread-1-processing-n:127.0.0.1:52458_solr) [n:127.0.0.1:52458_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 439703 INFO (zkConnectionManagerCallback-850-thread-1-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 439844 INFO (jetty-launcher-831-thread-2) [n:127.0.0.1:34724_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 439846 INFO (jetty-launcher-831-thread-2) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34724_solr
[junit4] 2> 439849 INFO (jetty-launcher-831-thread-2) [n:127.0.0.1:34724_solr ] o.a.s.c.Overseer Overseer (id=72231418660651014-127.0.0.1:34724_solr-n_0000000000) starting
[junit4] 2> 439861 INFO (jetty-launcher-831-thread-2) [n:127.0.0.1:34724_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34724_solr
[junit4] 2> 439864 INFO (OverseerStateUpdate-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 439877 INFO (jetty-launcher-831-thread-1) [n:127.0.0.1:52458_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 439879 INFO (jetty-launcher-831-thread-1) [n:127.0.0.1:52458_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 439884 INFO (jetty-launcher-831-thread-1) [n:127.0.0.1:52458_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52458_solr
[junit4] 2> 439889 INFO (zkCallback-849-thread-1-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 439890 INFO (zkCallback-847-thread-1-processing-n:127.0.0.1:52458_solr) [n:127.0.0.1:52458_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 440057 INFO (jetty-launcher-831-thread-2) [n:127.0.0.1:34724_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34724.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21d49731
[junit4] 2> 440062 INFO (jetty-launcher-831-thread-2) [n:127.0.0.1:34724_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34724.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21d49731
[junit4] 2> 440063 INFO (jetty-launcher-831-thread-2) [n:127.0.0.1:34724_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34724.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21d49731
[junit4] 2> 440063 INFO (jetty-launcher-831-thread-2) [n:127.0.0.1:34724_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/tempDir-001/node2/.
[junit4] 2> 440110 INFO (jetty-launcher-831-thread-1) [n:127.0.0.1:52458_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52458.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21d49731
[junit4] 2> 440117 INFO (jetty-launcher-831-thread-1) [n:127.0.0.1:52458_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52458.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21d49731
[junit4] 2> 440117 INFO (jetty-launcher-831-thread-1) [n:127.0.0.1:52458_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52458.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21d49731
[junit4] 2> 440119 INFO (jetty-launcher-831-thread-1) [n:127.0.0.1:52458_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/tempDir-001/node1/.
[junit4] 2> 440147 INFO (zkConnectionManagerCallback-856-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 440148 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1009e17b2e40007, likely client has closed socket
[junit4] 2> 440150 INFO (zkConnectionManagerCallback-860-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 440153 INFO (SUITE-AliasIntegrationTest-seed#[388583CDDAA1B4E0]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 440154 INFO (SUITE-AliasIntegrationTest-seed#[388583CDDAA1B4E0]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38847/solr ready
[junit4] 2> 440178 INFO (TEST-AliasIntegrationTest.testDeleteAliasWithExistingCollectionName-seed#[388583CDDAA1B4E0]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDeleteAliasWithExistingCollectionName
[junit4] 2> 440181 INFO (qtp208773353-3927) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection_old&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 440185 INFO (OverseerThreadFactory-1394-thread-1-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection_old
[junit4] 2> 440296 INFO (OverseerStateUpdate-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection_old",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection_old_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:52458/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 440299 INFO (OverseerStateUpdate-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection_old",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection_old_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34724/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 440502 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection_old_shard1_replica_n1&action=CREATE&numShards=2&collection=collection_old&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 440502 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 440503 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection_old_shard2_replica_n2&action=CREATE&numShards=2&collection=collection_old&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 440503 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 440616 INFO (zkCallback-849-thread-1-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 440616 INFO (zkCallback-847-thread-1-processing-n:127.0.0.1:52458_solr) [n:127.0.0.1:52458_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 440616 INFO (zkCallback-847-thread-2-processing-n:127.0.0.1:52458_solr) [n:127.0.0.1:52458_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 440616 INFO (zkCallback-849-thread-2-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441523 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 441523 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 441530 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.s.IndexSchema [collection_old_shard1_replica_n1] Schema name=minimal
[junit4] 2> 441532 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 441532 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection_old_shard1_replica_n1' using configuration from collection collection_old, trusted=true
[junit4] 2> 441532 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52458.solr.core.collection_old.shard1.replica_n1' (registry 'solr.core.collection_old.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21d49731
[junit4] 2> 441532 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 441532 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.SolrCore [[collection_old_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/tempDir-001/node1/collection_old_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/tempDir-001/node1/./collection_old_shard1_replica_n1/data/]
[junit4] 2> 441534 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.s.IndexSchema [collection_old_shard2_replica_n2] Schema name=minimal
[junit4] 2> 441543 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 441543 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection_old_shard2_replica_n2' using configuration from collection collection_old, trusted=true
[junit4] 2> 441544 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34724.solr.core.collection_old.shard2.replica_n2' (registry 'solr.core.collection_old.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21d49731
[junit4] 2> 441544 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 441544 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.SolrCore [[collection_old_shard2_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/tempDir-001/node2/collection_old_shard2_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/tempDir-001/node2/./collection_old_shard2_replica_n2/data/]
[junit4] 2> 441593 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 441593 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 441594 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 441594 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 441595 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@46e8feb0[collection_old_shard1_replica_n1] main]
[junit4] 2> 441595 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 441595 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 441595 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 441595 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592928575099502592
[junit4] 2> 441600 INFO (searcherExecutor-1399-thread-1-processing-n:127.0.0.1:52458_solr x:collection_old_shard1_replica_n1 s:shard1 c:collection_old r:core_node3) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.SolrCore [collection_old_shard1_replica_n1] Registered new searcher Searcher@46e8feb0[collection_old_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 441610 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 441610 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 441610 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52458/solr/collection_old_shard1_replica_n1/
[junit4] 2> 441610 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 441610 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:52458/solr/collection_old_shard1_replica_n1/ has no replicas
[junit4] 2> 441611 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 441616 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52458/solr/collection_old_shard1_replica_n1/ shard1
[junit4] 2> 441628 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 441628 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 441629 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 441629 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 441629 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3819dd51[collection_old_shard2_replica_n2] main]
[junit4] 2> 441630 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 441631 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 441632 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 441633 INFO (searcherExecutor-1400-thread-1-processing-n:127.0.0.1:34724_solr x:collection_old_shard2_replica_n2 s:shard2 c:collection_old r:core_node4) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.SolrCore [collection_old_shard2_replica_n2] Registered new searcher Searcher@3819dd51[collection_old_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 441633 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592928575139348480
[junit4] 2> 441641 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 441641 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 441641 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34724/solr/collection_old_shard2_replica_n2/
[junit4] 2> 441641 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 441641 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:34724/solr/collection_old_shard2_replica_n2/ has no replicas
[junit4] 2> 441641 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 441647 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34724/solr/collection_old_shard2_replica_n2/ shard2
[junit4] 2> 441749 INFO (zkCallback-847-thread-2-processing-n:127.0.0.1:52458_solr) [n:127.0.0.1:52458_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441749 INFO (zkCallback-847-thread-1-processing-n:127.0.0.1:52458_solr) [n:127.0.0.1:52458_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441749 INFO (zkCallback-849-thread-2-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441749 INFO (zkCallback-849-thread-1-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441767 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 441770 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection_old_shard1_replica_n1&action=CREATE&numShards=2&collection=collection_old&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1268
[junit4] 2> 441798 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 441802 INFO (qtp2030408775-3923) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection_old_shard2_replica_n2&action=CREATE&numShards=2&collection=collection_old&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1299
[junit4] 2> 441804 INFO (qtp208773353-3927) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 441902 INFO (zkCallback-849-thread-1-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441902 INFO (zkCallback-847-thread-2-processing-n:127.0.0.1:52458_solr) [n:127.0.0.1:52458_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441902 INFO (zkCallback-847-thread-1-processing-n:127.0.0.1:52458_solr) [n:127.0.0.1:52458_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441903 INFO (zkCallback-849-thread-2-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_old/state.json] for collection [collection_old] has occurred - updating... (live nodes size: [2])
[junit4] 2> 442186 INFO (OverseerCollectionConfigSetProcessor-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 442805 INFO (qtp208773353-3927) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection_old&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2625
[junit4] 2> 442806 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection_new&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 442809 INFO (OverseerThreadFactory-1394-thread-2-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection_new
[junit4] 2> 443020 INFO (OverseerStateUpdate-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection_new",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection_new_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34724/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 443221 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection_new_shard1_replica_n1&action=CREATE&numShards=1&collection=collection_new&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 443325 INFO (zkCallback-849-thread-2-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_new/state.json] for collection [collection_new] has occurred - updating... (live nodes size: [2])
[junit4] 2> 443325 INFO (zkCallback-849-thread-3-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_new/state.json] for collection [collection_new] has occurred - updating... (live nodes size: [2])
[junit4] 2> 444233 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 444239 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.s.IndexSchema [collection_new_shard1_replica_n1] Schema name=minimal
[junit4] 2> 444241 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 444241 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection_new_shard1_replica_n1' using configuration from collection collection_new, trusted=true
[junit4] 2> 444241 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34724.solr.core.collection_new.shard1.replica_n1' (registry 'solr.core.collection_new.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21d49731
[junit4] 2> 444241 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 444242 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.SolrCore [[collection_new_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/tempDir-001/node2/collection_new_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_388583CDDAA1B4E0-001/tempDir-001/node2/./collection_new_shard1_replica_n1/data/]
[junit4] 2> 444284 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 444284 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 444285 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 444285 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 444286 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@18960dd[collection_new_shard1_replica_n1] main]
[junit4] 2> 444287 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 444288 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 444288 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 444288 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592928577923317760
[junit4] 2> 444289 INFO (searcherExecutor-1409-thread-1-processing-n:127.0.0.1:34724_solr x:collection_new_shard1_replica_n1 s:shard1 c:collection_new r:core_node2) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.SolrCore [collection_new_shard1_replica_n1] Registered new searcher Searcher@18960dd[collection_new_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 444297 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 444297 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 444297 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34724/solr/collection_new_shard1_replica_n1/
[junit4] 2> 444297 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 444297 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:34724/solr/collection_new_shard1_replica_n1/ has no replicas
[junit4] 2> 444297 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 444302 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34724/solr/collection_new_shard1_replica_n1/ shard1
[junit4] 2> 444404 INFO (zkCallback-849-thread-3-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_new/state.json] for collection [collection_new] has occurred - updating... (live nodes size: [2])
[junit4] 2> 444404 INFO (zkCallback-849-thread-2-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_new/state.json] for collection [collection_new] has occurred - updating... (live nodes size: [2])
[junit4] 2> 444453 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 444454 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection_new_shard1_replica_n1&action=CREATE&numShards=1&collection=collection_new&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1234
[junit4] 2> 444457 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 444555 INFO (zkCallback-849-thread-3-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_new/state.json] for collection [collection_new] has occurred - updating... (live nodes size: [2])
[junit4] 2> 444555 INFO (zkCallback-849-thread-2-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_new/state.json] for collection [collection_new] has occurred - updating... (live nodes size: [2])
[junit4] 2> 444811 INFO (OverseerCollectionConfigSetProcessor-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 445458 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection_new&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2653
[junit4] 2> 445463 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection_old_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=collection_old:4&wt=javabin&version=2}{add=[8 (1592928579154345984)]} 0 1
[junit4] 2> 445464 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection_old_shard2_replica_n2] webapp=/solr path=/update params={_stateVer_=collection_old:4&wt=javabin&version=2}{add=[6 (1592928579154345984), 7 (1592928579156443136)]} 0 2
[junit4] 2> 445467 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1592928579159588864,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 445467 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@40c61f3f commitCommandVersion:1592928579159588864
[junit4] 2> 445468 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@25b82383[collection_old_shard1_replica_n1] main]
[junit4] 2> 445468 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 445469 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1592928579161686016,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 445469 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1320dd6c commitCommandVersion:1592928579161686016
[junit4] 2> 445470 INFO (searcherExecutor-1399-thread-1-processing-n:127.0.0.1:52458_solr x:collection_old_shard1_replica_n1 s:shard1 c:collection_old r:core_node3) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.SolrCore [collection_old_shard1_replica_n1] Registered new searcher Searcher@25b82383[collection_old_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):C1)))}
[junit4] 2> 445470 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection_old_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:34724/solr/collection_old_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 3
[junit4] 2> 445472 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@6e608ddc[collection_old_shard2_replica_n2] main]
[junit4] 2> 445472 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 445473 INFO (searcherExecutor-1400-thread-1-processing-n:127.0.0.1:34724_solr x:collection_old_shard2_replica_n2 s:shard2 c:collection_old r:core_node4) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.SolrCore [collection_old_shard2_replica_n2] Registered new searcher Searcher@6e608ddc[collection_old_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):C2)))}
[junit4] 2> 445473 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection_old_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:34724/solr/collection_old_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 4
[junit4] 2> 445473 INFO (qtp2030408775-3928) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection_old_shard2_replica_n2] webapp=/solr path=/update params={_stateVer_=collection_old:4&commit=true&wt=javabin&version=2}{commit=} 0 8
[junit4] 2> 445475 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection_new_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=collection_new:4&wt=javabin&version=2}{add=[1 (1592928579166928896)]} 0 0
[junit4] 2> 445476 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1592928579169026048,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 445477 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@7c6e5f9f commitCommandVersion:1592928579169026048
[junit4] 2> 445479 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6d3f76da[collection_new_shard1_replica_n1] main]
[junit4] 2> 445480 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 445480 INFO (searcherExecutor-1409-thread-1-processing-n:127.0.0.1:34724_solr x:collection_new_shard1_replica_n1 s:shard1 c:collection_new r:core_node2) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.SolrCore [collection_new_shard1_replica_n1] Registered new searcher Searcher@6d3f76da[collection_new_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):C1)))}
[junit4] 2> 445480 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection_new_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=collection_new:4&commit=true&wt=javabin&version=2}{commit=} 0 4
[junit4] 2> 445484 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.S.Request [collection_old_shard2_replica_n2] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=collection_old:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:34724/solr/collection_old_shard2_replica_n2/&rows=10&version=2&q=*:*&NOW=1519135073828&isShard=true&wt=javabin} hits=2 status=0 QTime=0
[junit4] 2> 445484 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.S.Request [collection_old_shard1_replica_n1] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=collection_old:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:52458/solr/collection_old_shard1_replica_n1/&rows=10&version=2&q=*:*&NOW=1519135073828&isShard=true&wt=javabin} hits=1 status=0 QTime=0
[junit4] 2> 445485 INFO (qtp208773353-3936) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.S.Request [collection_old_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&df=text&distrib=false&_stateVer_=collection_old:4&shards.purpose=64&NOW=1519135073828&ids=8&isShard=true&shard.url=http://127.0.0.1:52458/solr/collection_old_shard1_replica_n1/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 445485 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_old s:shard2 r:core_node4 x:collection_old_shard2_replica_n2] o.a.s.c.S.Request [collection_old_shard2_replica_n2] webapp=/solr path=/select params={q=*:*&df=text&distrib=false&_stateVer_=collection_old:4&shards.purpose=64&NOW=1519135073828&ids=6,7&isShard=true&shard.url=http://127.0.0.1:34724/solr/collection_old_shard2_replica_n2/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 445485 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr c:collection_old s:shard1 r:core_node3 x:collection_old_shard1_replica_n1] o.a.s.c.S.Request [collection_old_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=collection_old:4&wt=javabin&version=2} hits=3 status=0 QTime=4
[junit4] 2> 445488 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createalias with params collections=collection_old&name=collection_old_reserve&action=CREATEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 445599 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collections=collection_old&name=collection_old_reserve&action=CREATEALIAS&wt=javabin&version=2} status=0 QTime=111
[junit4] 2> 445600 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createalias with params collections=collection_new&name=collection_old&action=CREATEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 445602 INFO (OverseerCollectionConfigSetProcessor-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 445705 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collections=collection_new&name=collection_old&action=CREATEALIAS&wt=javabin&version=2} status=0 QTime=105
[junit4] 2> 445706 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.S.Request [collection_new_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=collection_old:4&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 445707 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.S.Request [collection_new_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=collection_new:4&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 445707 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params async=c6694513-cd8b-4430-9bcf-ce52213c0e13&name=collection_old&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 445710 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=c6694513-cd8b-4430-9bcf-ce52213c0e13&name=collection_old&action=DELETE&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 445711 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=c6694513-cd8b-4430-9bcf-ce52213c0e13&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 445712 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=c6694513-cd8b-4430-9bcf-ce52213c0e13&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 445714 INFO (OverseerCollectionConfigSetProcessor-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 445714 ERROR (OverseerThreadFactory-1394-thread-5-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: collection_old operation: delete failed:org.apache.solr.common.SolrException: Collection : collection_old is part of alias collection_old_reserve remove or modify the alias before removing this collection.
[junit4] 2> at org.apache.solr.cloud.api.collections.DeleteCollectionCmd.call(DeleteCollectionCmd.java:71)
[junit4] 2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:254)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 445723 INFO (OverseerThreadFactory-1394-thread-5-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 446713 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=c6694513-cd8b-4430-9bcf-ce52213c0e13&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 446715 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=c6694513-cd8b-4430-9bcf-ce52213c0e13&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 446716 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=c6694513-cd8b-4430-9bcf-ce52213c0e13&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 446717 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=c6694513-cd8b-4430-9bcf-ce52213c0e13&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 446720 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr c:collection_new s:shard1 r:core_node2 x:collection_new_shard1_replica_n1] o.a.s.c.S.Request [collection_new_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=collection_old:4&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 446722 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletealias with params async=bed00b9d-d0e6-4183-81cb-a0b7f34d3823&name=collection_old_reserve&action=DELETEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 446725 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=bed00b9d-d0e6-4183-81cb-a0b7f34d3823&name=collection_old_reserve&action=DELETEALIAS&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 446725 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=bed00b9d-d0e6-4183-81cb-a0b7f34d3823&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 446726 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=bed00b9d-d0e6-4183-81cb-a0b7f34d3823&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 446729 INFO (OverseerCollectionConfigSetProcessor-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 446779 INFO (OverseerThreadFactory-1394-thread-5-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000009 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 447728 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=bed00b9d-d0e6-4183-81cb-a0b7f34d3823&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 447729 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=bed00b9d-d0e6-4183-81cb-a0b7f34d3823&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 447730 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=bed00b9d-d0e6-4183-81cb-a0b7f34d3823&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 447730 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=bed00b9d-d0e6-4183-81cb-a0b7f34d3823&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 447731 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletealias with params async=eeb384f0-7f92-4e7d-a7b9-ee2120a6d410&name=collection_old&action=DELETEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 447734 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=eeb384f0-7f92-4e7d-a7b9-ee2120a6d410&name=collection_old&action=DELETEALIAS&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 447735 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=eeb384f0-7f92-4e7d-a7b9-ee2120a6d410&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 447737 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eeb384f0-7f92-4e7d-a7b9-ee2120a6d410&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 447738 INFO (OverseerCollectionConfigSetProcessor-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000009 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 447741 INFO (OverseerThreadFactory-1394-thread-5-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 448739 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=eeb384f0-7f92-4e7d-a7b9-ee2120a6d410&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 448740 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eeb384f0-7f92-4e7d-a7b9-ee2120a6d410&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 448741 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=eeb384f0-7f92-4e7d-a7b9-ee2120a6d410&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 448741 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eeb384f0-7f92-4e7d-a7b9-ee2120a6d410&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 448742 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params async=e7bf5602-ecde-4f79-ae5e-49424ea45a93&name=collection_new&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 448757 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=e7bf5602-ecde-4f79-ae5e-49424ea45a93&name=collection_new&action=DELETE&wt=javabin&version=2} status=0 QTime=14
[junit4] 2> 448757 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=e7bf5602-ecde-4f79-ae5e-49424ea45a93&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 448759 INFO (OverseerThreadFactory-1394-thread-5-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=e7bf5602-ecde-4f79-ae5e-49424ea45a93
[junit4] 2> 448760 INFO (OverseerCollectionConfigSetProcessor-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 448760 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=e7bf5602-ecde-4f79-ae5e-49424ea45a93&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 448761 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection_new_shard1_replica_n1&async=e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 448761 INFO (parallelCoreAdminExecutor-1395-thread-1-processing-n:127.0.0.1:34724_solr e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942 UNLOAD) [n:127.0.0.1:34724_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection_new.shard1.replica_n1, tag=null
[junit4] 2> 448762 INFO (parallelCoreAdminExecutor-1395-thread-1-processing-n:127.0.0.1:34724_solr e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942 UNLOAD) [n:127.0.0.1:34724_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4c2d92e8: rootName = solr_34724, domain = solr.core.collection_new.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection_new.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@6ca29ff0
[junit4] 2> 448762 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942
[junit4] 2> 448762 INFO (qtp2030408775-3925) [n:127.0.0.1:34724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 448768 INFO (parallelCoreAdminExecutor-1395-thread-1-processing-n:127.0.0.1:34724_solr e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942 UNLOAD) [n:127.0.0.1:34724_solr ] o.a.s.c.SolrCore [collection_new_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@4c088614
[junit4] 2> 448768 INFO (parallelCoreAdminExecutor-1395-thread-1-processing-n:127.0.0.1:34724_solr e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942 UNLOAD) [n:127.0.0.1:34724_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection_new.shard1.replica_n1, tag=1275627028
[junit4] 2> 448768 INFO (parallelCoreAdminExecutor-1395-thread-1-processing-n:127.0.0.1:34724_solr e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942 UNLOAD) [n:127.0.0.1:34724_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection_new.shard1.leader, tag=1275627028
[junit4] 2> 449767 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=e7bf5602-ecde-4f79-ae5e-49424ea45a93&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 449769 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=e7bf5602-ecde-4f79-ae5e-49424ea45a93&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 449770 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942
[junit4] 2> 449770 INFO (qtp2030408775-3930) [n:127.0.0.1:34724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=e7bf5602-ecde-4f79-ae5e-49424ea45a932652362454907942&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 450484 INFO (OverseerThreadFactory-1394-thread-5-processing-n:127.0.0.1:34724_solr) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000011 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 450761 INFO (OverseerCollectionConfigSetProcessor-72231418660651014-127.0.0.1:34724_solr-n_0000000000) [n:127.0.0.1:34724_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000011 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 450770 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=e7bf5602-ecde-4f79-ae5e-49424ea45a93&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 450772 INFO (qtp208773353-3937) [n:127.0.0.1:52458_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=e7bf5602-ecde-4f79-ae5e-49424ea45a93&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 450773 INFO (qtp208773353-3934) [n:127.0.0.1:52458_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=e7bf5602-ecde-4f79-ae5e-49424ea45a93&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 450774 INFO (qtp208773353-3934) [n:
[...truncated too long message...]
rol tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=14459, name=jetty-launcher-2878-thread-2-SendThread(127.0.0.1:56087), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=449, maxMBSortInHeap=5.480277735019612, sim=RandomSimilarity(queryNorm=true): {}, locale=nn-NO, timezone=Africa/Ndjamena
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=4,free=202828616,total=444596224
[junit4] 2> NOTE: All tests run in this JVM: [TestPayloadScoreQParserPlugin, TestComputePlanAction, TestCloudRecovery, TestSolrCoreSnapshots, TestSegmentSorting, CdcrReplicationHandlerTest, LoggingHandlerTest, TestUseDocValuesAsStored2, TestCloudPivotFacet, TestLuceneMatchVersion, TestFastLRUCache, TestComplexPhraseQParserPlugin, CSVRequestHandlerTest, DeleteStatusTest, ReplaceNodeTest, TestCollectionsAPIViaSolrCloudCluster, TestAtomicUpdateErrorCases, TestXmlQParser, URLClassifyProcessorTest, HdfsAutoAddReplicasIntegrationTest, ReturnFieldsTest, TestClusterStateMutator, DocExpirationUpdateProcessorFactoryTest, TestDistribIDF, TestWriterPerf, TestDistributedMissingSort, TestInPlaceUpdatesStandalone, TestReRankQParserPlugin, UninvertDocValuesMergePolicyTest, HdfsWriteToMultipleCollectionsTest, V2ApiIntegrationTest, ClassificationUpdateProcessorFactoryTest, TestPKIAuthenticationPlugin, CollectionReloadTest, PrimitiveFieldTypeTest, AliasIntegrationTest, UUIDFieldTest, TestLeaderInitiatedRecoveryThread, EchoParamsTest, CleanupOldIndexTest, TestNumericRangeQuery32, JvmMetricsTest, QueryParsingTest, TestFieldCacheSort, ZkShardTermsTest, TestExtendedDismaxParser, TestRestoreCore, LeaderFailoverAfterPartitionTest, TestStressVersions, TestDistributedSearch, BadIndexSchemaTest, TestSerializedLuceneMatchVersion, MoveReplicaHDFSFailoverTest, ChaosMonkeySafeLeaderTest, SharedFSAutoReplicaFailoverTest, TestRandomDVFaceting, ConnectionManagerTest, ZkControllerTest, TestMiniSolrCloudClusterSSL, TriggerIntegrationTest, SolrSlf4jReporterTest, CdcrVersionReplicationTest, TestRequestForwarding, TestLocalFSCloudBackupRestore, UpdateRequestProcessorFactoryTest, TestNumericTerms64, TestBinaryResponseWriter, OverriddenZkACLAndCredentialsProvidersTest, TlogReplayBufferedWhileIndexingTest, SimpleCollectionCreateDeleteTest, SoftAutoCommitTest, TestSimpleQParserPlugin, TestLegacyFieldCache, DistributedTermsComponentTest, TestCloudInspectUtil, MoveReplicaTest, SolrCoreCheckLockOnStartupTest, ResponseLogComponentTest, TestInitQParser, TestMaxTokenLenTokenizer, DocValuesMissingTest, TestInitParams, TestScoreJoinQPScore, SSLMigrationTest, TestBlobHandler, TestReversedWildcardFilterFactory, TermVectorComponentTest, TestClassicSimilarityFactory, SolrMetricReporterTest, TestGraphMLResponseWriter, HdfsCollectionsAPIDistributedZkTest, SpatialHeatmapFacetsTest, ChaosMonkeyNothingIsSafeTest, BlockCacheTest, HdfsChaosMonkeyNothingIsSafeTest, CustomHighlightComponentTest, TestLMDirichletSimilarityFactory, TestSha256AuthenticationProvider, TestJettySolrRunner, BufferStoreTest, TestSchemaNameResource, ScriptEngineTest, TestDistributedGrouping, ClusterStateTest, BasicAuthIntegrationTest, FileBasedSpellCheckerTest, DistributedSpellCheckComponentTest, AutoscalingHistoryHandlerTest, TestSimpleTrackingShardHandler, TestBulkSchemaAPI, TestQuerySenderListener, SpellPossibilityIteratorTest, HdfsLockFactoryTest, TestValueSourceCache, TestUpdate, TestSolrIndexConfig, TestFuzzyAnalyzedSuggestions, TestMultiWordSynonyms, RegexBoostProcessorTest, IndexBasedSpellCheckerTest, TestPullReplicaErrorHandling, SliceStateTest, RestartWhileUpdatingTest, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=388583CDDAA1B4E0 -Dtests.slow=true -Dtests.locale=nn-NO -Dtests.timezone=Africa/Ndjamena -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=14459, name=jetty-launcher-2878-thread-2-SendThread(127.0.0.1:56087), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > 2) Thread[id=14460, name=jetty-launcher-2878-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4] > 3) Thread[id=14392, name=jetty-launcher-2878-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
[junit4] > at __randomizedtesting.SeedInfo.seed([388583CDDAA1B4E0]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=14459, name=jetty-launcher-2878-thread-2-SendThread(127.0.0.1:56087), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > at __randomizedtesting.SeedInfo.seed([388583CDDAA1B4E0]:0)
[junit4] Completed [284/777 (3!)] on J1 in 64.35s, 12 tests, 2 errors <<< FAILURES!
[...truncated 10848 lines...]
-validate-source-patterns:
[source-patterns] invalid logging pattern [not private static final, uses static class name]: solr/contrib/prometheus-exporter/src/java/org/apache/solr/prometheus/collector/SolrCollector.java
[source-patterns] invalid logging pattern [not private static final, uses static class name]: solr/contrib/prometheus-exporter/src/java/org/apache/solr/prometheus/exporter/SolrExporter.java
[source-patterns] invalid logging pattern [not private static final, uses static class name]: solr/contrib/prometheus-exporter/src/java/org/apache/solr/prometheus/scraper/SolrScraper.java
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/build.xml:618: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/build.xml:128: Found 3 violations in source files (invalid logging pattern [not private static final, uses static class name]).
Total time: 73 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any