You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/10/22 00:58:46 UTC

[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 412 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/412/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:40168/k/j/collection1 Dead Guy:http://127.0.0.1:37639/k/j/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:32947/k/j/collection1], CloudJettyRunner [url=http://127.0.0.1:32947/k/j/collection1]]

Stack Trace:
java.lang.AssertionError: Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:40168/k/j/collection1 Dead Guy:http://127.0.0.1:37639/k/j/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:32947/k/j/collection1], CloudJettyRunner [url=http://127.0.0.1:32947/k/j/collection1]]
	at __randomizedtesting.SeedInfo.seed([5FBFF3A3C3A12E9A:DE597DBBB4FE4EA6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:216)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:679)


REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1382394204821,generation=2,filelist=[_85.cfe, _85.cfs, _85.si, _87.fdt, _87.fdx, _87.fnm, _87.si, _87_Lucene41_0.doc, _87_Lucene41_0.tim, _87_Lucene41_0.tip, _87_nrm.cfe, _87_nrm.cfs, _88.fdt, _88.fdx, _88.fnm, _88.si, _88_Lucene41_0.doc, _88_Lucene41_0.tim, _88_Lucene41_0.tip, _88_nrm.cfe, _88_nrm.cfs, _8a.fdt, _8a.fdx, _8a.fnm, _8a.si, _8a_Lucene41_0.doc, _8a_Lucene41_0.tim, _8a_Lucene41_0.tip, _8a_nrm.cfe, _8a_nrm.cfs, _8d.fdt, _8d.fdx, _8d.fnm, _8d.si, _8d_Lucene41_0.doc, _8d_Lucene41_0.tim, _8d_Lucene41_0.tip, _8d_nrm.cfe, _8d_nrm.cfs, _8g.fdt, _8g.fdx, _8g.fnm, _8g.si, _8g_Lucene41_0.doc, _8g_Lucene41_0.tim, _8g_Lucene41_0.tip, _8g_nrm.cfe, _8g_nrm.cfs, _8j.fdt, _8j.fdx, _8j.fnm, _8j.si, _8j_Lucene41_0.doc, _8j_Lucene41_0.tim, _8j_Lucene41_0.tip, _8j_nrm.cfe, _8j_nrm.cfs, _8l.fdt, _8l.fdx, _8l.fnm, _8l.si, _8l_Lucene41_0.doc, _8l_Lucene41_0.tim, _8l_Lucene41_0.tip, _8l_nrm.cfe, _8l_nrm.cfs, _8m.cfe, _8m.cfs, _8m.si, _8n.fdt, _8n.fdx, _8n.fnm, _8n.si, _8n_Lucene41_0.doc, _8n_Lucene41_0.tim, _8n_Lucene41_0.tip, _8n_nrm.cfe, _8n_nrm.cfs, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.si, _8p_Lucene41_0.doc, _8p_Lucene41_0.tim, _8p_Lucene41_0.tip, _8p_nrm.cfe, _8p_nrm.cfs, _8q.fdt, _8q.fdx, _8q.fnm, _8q.si, _8q_Lucene41_0.doc, _8q_Lucene41_0.tim, _8q_Lucene41_0.tip, _8q_nrm.cfe, _8q_nrm.cfs, _8r.fdt, _8r.fdx, _8r.fnm, _8r.si, _8r_Lucene41_0.doc, _8r_Lucene41_0.tim, _8r_Lucene41_0.tip, _8r_nrm.cfe, _8r_nrm.cfs, _8s.fdt, _8s.fdx, _8s.fnm, _8s.si, _8s_Lucene41_0.doc, _8s_Lucene41_0.tim, _8s_Lucene41_0.tip, _8s_nrm.cfe, _8s_nrm.cfs, _8t.fdt, _8t.fdx, _8t.fnm, _8t.si, _8t_Lucene41_0.doc, _8t_Lucene41_0.tim, _8t_Lucene41_0.tip, _8t_nrm.cfe, _8t_nrm.cfs, _8u.fdt, _8u.fdx, _8u.fnm, _8u.si, _8u_Lucene41_0.doc, _8u_Lucene41_0.tim, _8u_Lucene41_0.tip, _8u_nrm.cfe, _8u_nrm.cfs, _8v.fdt, _8v.fdx, _8v.fnm, _8v.si, _8v_Lucene41_0.doc, _8v_Lucene41_0.tim, _8v_Lucene41_0.tip, _8v_nrm.cfe, _8v_nrm.cfs, segments_2]}]> but was:<[{indexVersion=1382394204821,generation=2,filelist=[_85.cfe, _85.cfs, _85.si, _87.fdt, _87.fdx, _87.fnm, _87.si, _87_Lucene41_0.doc, _87_Lucene41_0.tim, _87_Lucene41_0.tip, _87_nrm.cfe, _87_nrm.cfs, _88.fdt, _88.fdx, _88.fnm, _88.si, _88_Lucene41_0.doc, _88_Lucene41_0.tim, _88_Lucene41_0.tip, _88_nrm.cfe, _88_nrm.cfs, _8a.fdt, _8a.fdx, _8a.fnm, _8a.si, _8a_Lucene41_0.doc, _8a_Lucene41_0.tim, _8a_Lucene41_0.tip, _8a_nrm.cfe, _8a_nrm.cfs, _8d.fdt, _8d.fdx, _8d.fnm, _8d.si, _8d_Lucene41_0.doc, _8d_Lucene41_0.tim, _8d_Lucene41_0.tip, _8d_nrm.cfe, _8d_nrm.cfs, _8g.fdt, _8g.fdx, _8g.fnm, _8g.si, _8g_Lucene41_0.doc, _8g_Lucene41_0.tim, _8g_Lucene41_0.tip, _8g_nrm.cfe, _8g_nrm.cfs, _8j.fdt, _8j.fdx, _8j.fnm, _8j.si, _8j_Lucene41_0.doc, _8j_Lucene41_0.tim, _8j_Lucene41_0.tip, _8j_nrm.cfe, _8j_nrm.cfs, _8l.fdt, _8l.fdx, _8l.fnm, _8l.si, _8l_Lucene41_0.doc, _8l_Lucene41_0.tim, _8l_Lucene41_0.tip, _8l_nrm.cfe, _8l_nrm.cfs, _8m.cfe, _8m.cfs, _8m.si, _8n.fdt, _8n.fdx, _8n.fnm, _8n.si, _8n_Lucene41_0.doc, _8n_Lucene41_0.tim, _8n_Lucene41_0.tip, _8n_nrm.cfe, _8n_nrm.cfs, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.si, _8p_Lucene41_0.doc, _8p_Lucene41_0.tim, _8p_Lucene41_0.tip, _8p_nrm.cfe, _8p_nrm.cfs, _8q.fdt, _8q.fdx, _8q.fnm, _8q.si, _8q_Lucene41_0.doc, _8q_Lucene41_0.tim, _8q_Lucene41_0.tip, _8q_nrm.cfe, _8q_nrm.cfs, _8r.fdt, _8r.fdx, _8r.fnm, _8r.si, _8r_Lucene41_0.doc, _8r_Lucene41_0.tim, _8r_Lucene41_0.tip, _8r_nrm.cfe, _8r_nrm.cfs, _8s.fdt, _8s.fdx, _8s.fnm, _8s.si, _8s_Lucene41_0.doc, _8s_Lucene41_0.tim, _8s_Lucene41_0.tip, _8s_nrm.cfe, _8s_nrm.cfs, _8t.fdt, _8t.fdx, _8t.fnm, _8t.si, _8t_Lucene41_0.doc, _8t_Lucene41_0.tim, _8t_Lucene41_0.tip, _8t_nrm.cfe, _8t_nrm.cfs, _8u.fdt, _8u.fdx, _8u.fnm, _8u.si, _8u_Lucene41_0.doc, _8u_Lucene41_0.tim, _8u_Lucene41_0.tip, _8u_nrm.cfe, _8u_nrm.cfs, _8v.fdt, _8v.fdx, _8v.fnm, _8v.si, _8v_Lucene41_0.doc, _8v_Lucene41_0.tim, _8v_Lucene41_0.tip, _8v_nrm.cfe, _8v_nrm.cfs, segments_2]}, {indexVersion=1382394204821,generation=3,filelist=[_8a.fdt, _8a.fdx, _8a.fnm, _8a.si, _8a_Lucene41_0.doc, _8a_Lucene41_0.tim, _8a_Lucene41_0.tip, _8a_nrm.cfe, _8a_nrm.cfs, _8d.fdt, _8d.fdx, _8d.fnm, _8d.si, _8d_Lucene41_0.doc, _8d_Lucene41_0.tim, _8d_Lucene41_0.tip, _8d_nrm.cfe, _8d_nrm.cfs, _8n.fdt, _8n.fdx, _8n.fnm, _8n.si, _8n_Lucene41_0.doc, _8n_Lucene41_0.tim, _8n_Lucene41_0.tip, _8n_nrm.cfe, _8n_nrm.cfs, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.si, _8p_Lucene41_0.doc, _8p_Lucene41_0.tim, _8p_Lucene41_0.tip, _8p_nrm.cfe, _8p_nrm.cfs, _8q.fdt, _8q.fdx, _8q.fnm, _8q.si, _8q_Lucene41_0.doc, _8q_Lucene41_0.tim, _8q_Lucene41_0.tip, _8q_nrm.cfe, _8q_nrm.cfs, _8r.fdt, _8r.fdx, _8r.fnm, _8r.si, _8r_Lucene41_0.doc, _8r_Lucene41_0.tim, _8r_Lucene41_0.tip, _8r_nrm.cfe, _8r_nrm.cfs, _8u.fdt, _8u.fdx, _8u.fnm, _8u.si, _8u_Lucene41_0.doc, _8u_Lucene41_0.tim, _8u_Lucene41_0.tip, _8u_nrm.cfe, _8u_nrm.cfs, _8v.fdt, _8v.fdx, _8v.fnm, _8v.si, _8v_Lucene41_0.doc, _8v_Lucene41_0.tim, _8v_Lucene41_0.tip, _8v_nrm.cfe, _8v_nrm.cfs, _8w.cfe, _8w.cfs, _8w.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1382394204821,generation=2,filelist=[_85.cfe, _85.cfs, _85.si, _87.fdt, _87.fdx, _87.fnm, _87.si, _87_Lucene41_0.doc, _87_Lucene41_0.tim, _87_Lucene41_0.tip, _87_nrm.cfe, _87_nrm.cfs, _88.fdt, _88.fdx, _88.fnm, _88.si, _88_Lucene41_0.doc, _88_Lucene41_0.tim, _88_Lucene41_0.tip, _88_nrm.cfe, _88_nrm.cfs, _8a.fdt, _8a.fdx, _8a.fnm, _8a.si, _8a_Lucene41_0.doc, _8a_Lucene41_0.tim, _8a_Lucene41_0.tip, _8a_nrm.cfe, _8a_nrm.cfs, _8d.fdt, _8d.fdx, _8d.fnm, _8d.si, _8d_Lucene41_0.doc, _8d_Lucene41_0.tim, _8d_Lucene41_0.tip, _8d_nrm.cfe, _8d_nrm.cfs, _8g.fdt, _8g.fdx, _8g.fnm, _8g.si, _8g_Lucene41_0.doc, _8g_Lucene41_0.tim, _8g_Lucene41_0.tip, _8g_nrm.cfe, _8g_nrm.cfs, _8j.fdt, _8j.fdx, _8j.fnm, _8j.si, _8j_Lucene41_0.doc, _8j_Lucene41_0.tim, _8j_Lucene41_0.tip, _8j_nrm.cfe, _8j_nrm.cfs, _8l.fdt, _8l.fdx, _8l.fnm, _8l.si, _8l_Lucene41_0.doc, _8l_Lucene41_0.tim, _8l_Lucene41_0.tip, _8l_nrm.cfe, _8l_nrm.cfs, _8m.cfe, _8m.cfs, _8m.si, _8n.fdt, _8n.fdx, _8n.fnm, _8n.si, _8n_Lucene41_0.doc, _8n_Lucene41_0.tim, _8n_Lucene41_0.tip, _8n_nrm.cfe, _8n_nrm.cfs, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.si, _8p_Lucene41_0.doc, _8p_Lucene41_0.tim, _8p_Lucene41_0.tip, _8p_nrm.cfe, _8p_nrm.cfs, _8q.fdt, _8q.fdx, _8q.fnm, _8q.si, _8q_Lucene41_0.doc, _8q_Lucene41_0.tim, _8q_Lucene41_0.tip, _8q_nrm.cfe, _8q_nrm.cfs, _8r.fdt, _8r.fdx, _8r.fnm, _8r.si, _8r_Lucene41_0.doc, _8r_Lucene41_0.tim, _8r_Lucene41_0.tip, _8r_nrm.cfe, _8r_nrm.cfs, _8s.fdt, _8s.fdx, _8s.fnm, _8s.si, _8s_Lucene41_0.doc, _8s_Lucene41_0.tim, _8s_Lucene41_0.tip, _8s_nrm.cfe, _8s_nrm.cfs, _8t.fdt, _8t.fdx, _8t.fnm, _8t.si, _8t_Lucene41_0.doc, _8t_Lucene41_0.tim, _8t_Lucene41_0.tip, _8t_nrm.cfe, _8t_nrm.cfs, _8u.fdt, _8u.fdx, _8u.fnm, _8u.si, _8u_Lucene41_0.doc, _8u_Lucene41_0.tim, _8u_Lucene41_0.tip, _8u_nrm.cfe, _8u_nrm.cfs, _8v.fdt, _8v.fdx, _8v.fnm, _8v.si, _8v_Lucene41_0.doc, _8v_Lucene41_0.tim, _8v_Lucene41_0.tip, _8v_nrm.cfe, _8v_nrm.cfs, segments_2]}]> but was:<[{indexVersion=1382394204821,generation=2,filelist=[_85.cfe, _85.cfs, _85.si, _87.fdt, _87.fdx, _87.fnm, _87.si, _87_Lucene41_0.doc, _87_Lucene41_0.tim, _87_Lucene41_0.tip, _87_nrm.cfe, _87_nrm.cfs, _88.fdt, _88.fdx, _88.fnm, _88.si, _88_Lucene41_0.doc, _88_Lucene41_0.tim, _88_Lucene41_0.tip, _88_nrm.cfe, _88_nrm.cfs, _8a.fdt, _8a.fdx, _8a.fnm, _8a.si, _8a_Lucene41_0.doc, _8a_Lucene41_0.tim, _8a_Lucene41_0.tip, _8a_nrm.cfe, _8a_nrm.cfs, _8d.fdt, _8d.fdx, _8d.fnm, _8d.si, _8d_Lucene41_0.doc, _8d_Lucene41_0.tim, _8d_Lucene41_0.tip, _8d_nrm.cfe, _8d_nrm.cfs, _8g.fdt, _8g.fdx, _8g.fnm, _8g.si, _8g_Lucene41_0.doc, _8g_Lucene41_0.tim, _8g_Lucene41_0.tip, _8g_nrm.cfe, _8g_nrm.cfs, _8j.fdt, _8j.fdx, _8j.fnm, _8j.si, _8j_Lucene41_0.doc, _8j_Lucene41_0.tim, _8j_Lucene41_0.tip, _8j_nrm.cfe, _8j_nrm.cfs, _8l.fdt, _8l.fdx, _8l.fnm, _8l.si, _8l_Lucene41_0.doc, _8l_Lucene41_0.tim, _8l_Lucene41_0.tip, _8l_nrm.cfe, _8l_nrm.cfs, _8m.cfe, _8m.cfs, _8m.si, _8n.fdt, _8n.fdx, _8n.fnm, _8n.si, _8n_Lucene41_0.doc, _8n_Lucene41_0.tim, _8n_Lucene41_0.tip, _8n_nrm.cfe, _8n_nrm.cfs, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.si, _8p_Lucene41_0.doc, _8p_Lucene41_0.tim, _8p_Lucene41_0.tip, _8p_nrm.cfe, _8p_nrm.cfs, _8q.fdt, _8q.fdx, _8q.fnm, _8q.si, _8q_Lucene41_0.doc, _8q_Lucene41_0.tim, _8q_Lucene41_0.tip, _8q_nrm.cfe, _8q_nrm.cfs, _8r.fdt, _8r.fdx, _8r.fnm, _8r.si, _8r_Lucene41_0.doc, _8r_Lucene41_0.tim, _8r_Lucene41_0.tip, _8r_nrm.cfe, _8r_nrm.cfs, _8s.fdt, _8s.fdx, _8s.fnm, _8s.si, _8s_Lucene41_0.doc, _8s_Lucene41_0.tim, _8s_Lucene41_0.tip, _8s_nrm.cfe, _8s_nrm.cfs, _8t.fdt, _8t.fdx, _8t.fnm, _8t.si, _8t_Lucene41_0.doc, _8t_Lucene41_0.tim, _8t_Lucene41_0.tip, _8t_nrm.cfe, _8t_nrm.cfs, _8u.fdt, _8u.fdx, _8u.fnm, _8u.si, _8u_Lucene41_0.doc, _8u_Lucene41_0.tim, _8u_Lucene41_0.tip, _8u_nrm.cfe, _8u_nrm.cfs, _8v.fdt, _8v.fdx, _8v.fnm, _8v.si, _8v_Lucene41_0.doc, _8v_Lucene41_0.tim, _8v_Lucene41_0.tip, _8v_nrm.cfe, _8v_nrm.cfs, segments_2]}, {indexVersion=1382394204821,generation=3,filelist=[_8a.fdt, _8a.fdx, _8a.fnm, _8a.si, _8a_Lucene41_0.doc, _8a_Lucene41_0.tim, _8a_Lucene41_0.tip, _8a_nrm.cfe, _8a_nrm.cfs, _8d.fdt, _8d.fdx, _8d.fnm, _8d.si, _8d_Lucene41_0.doc, _8d_Lucene41_0.tim, _8d_Lucene41_0.tip, _8d_nrm.cfe, _8d_nrm.cfs, _8n.fdt, _8n.fdx, _8n.fnm, _8n.si, _8n_Lucene41_0.doc, _8n_Lucene41_0.tim, _8n_Lucene41_0.tip, _8n_nrm.cfe, _8n_nrm.cfs, _8o.cfe, _8o.cfs, _8o.si, _8p.fdt, _8p.fdx, _8p.fnm, _8p.si, _8p_Lucene41_0.doc, _8p_Lucene41_0.tim, _8p_Lucene41_0.tip, _8p_nrm.cfe, _8p_nrm.cfs, _8q.fdt, _8q.fdx, _8q.fnm, _8q.si, _8q_Lucene41_0.doc, _8q_Lucene41_0.tim, _8q_Lucene41_0.tip, _8q_nrm.cfe, _8q_nrm.cfs, _8r.fdt, _8r.fdx, _8r.fnm, _8r.si, _8r_Lucene41_0.doc, _8r_Lucene41_0.tim, _8r_Lucene41_0.tip, _8r_nrm.cfe, _8r_nrm.cfs, _8u.fdt, _8u.fdx, _8u.fnm, _8u.si, _8u_Lucene41_0.doc, _8u_Lucene41_0.tim, _8u_Lucene41_0.tip, _8u_nrm.cfe, _8u_nrm.cfs, _8v.fdt, _8v.fdx, _8v.fnm, _8v.si, _8v_Lucene41_0.doc, _8v_Lucene41_0.tim, _8v_Lucene41_0.tip, _8v_nrm.cfe, _8v_nrm.cfs, _8w.cfe, _8w.cfs, _8w.si, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([5FBFF3A3C3A12E9A:7A68E893B3E92099]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:679)




Build Log:
[...truncated 9451 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> 104827 T503 oas.SolrTestCaseJ4.setUp ###Starting doTestDetails
   [junit4]   2> 105784 T503 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 106260 T503 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:39548
   [junit4]   2> 106334 T503 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 106335 T503 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 106335 T503 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master
   [junit4]   2> 106336 T503 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/'
   [junit4]   2> 106376 T503 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/solr.xml
   [junit4]   2> 106462 T503 oasc.CoreContainer.<init> New CoreContainer 2078185786
   [junit4]   2> 106463 T503 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/]
   [junit4]   2> 106539 T503 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 106539 T503 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 106539 T503 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 106540 T503 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 106540 T503 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 106540 T503 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 106541 T503 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 106541 T503 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 106541 T503 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 106547 T503 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 106547 T503 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 106552 T515 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1
   [junit4]   2> 106553 T515 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/'
   [junit4]   2> 106597 T515 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
   [junit4]   2> 106641 T515 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 106642 T515 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 106647 T515 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 106668 T515 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 106669 T515 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 106670 T515 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data/
   [junit4]   2> 106670 T515 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 106671 T515 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data
   [junit4]   2> 106672 T515 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data/index/
   [junit4]   2> 106672 T515 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 106672 T515 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data/index
   [junit4]   2> 106673 T515 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1531038594, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 106674 T515 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_1,generation=1}
   [junit4]   2> 106675 T515 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 106675 T515 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 106676 T515 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 106726 T515 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 106726 T515 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 106726 T515 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 106726 T515 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 106727 T515 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 106733 T515 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 106738 T515 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 106738 T515 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 106738 T515 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@56af4653
   [junit4]   2> 106739 T515 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_1,generation=1}
   [junit4]   2> 106739 T515 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 106740 T515 oass.SolrIndexSearcher.<init> Opening Searcher@32985ae9 main
   [junit4]   2> 106744 T515 oash.ReplicationHandler.inform Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 106748 T515 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 106748 T516 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@32985ae9 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 106748 T515 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 106749 T503 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
   [junit4]   2> 106749 T503 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 106840 T503 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 106842 T503 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:51011
   [junit4]   2> 106843 T503 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 106843 T503 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 106844 T503 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave
   [junit4]   2> 106844 T503 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/'
   [junit4]   2> 106880 T503 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/solr.xml
   [junit4]   2> 106935 T503 oasc.CoreContainer.<init> New CoreContainer 1950058271
   [junit4]   2> 106936 T503 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/]
   [junit4]   2> 106937 T503 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 106937 T503 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 106937 T503 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 106938 T503 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 106938 T503 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 106938 T503 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 106939 T503 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 106939 T503 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 106939 T503 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 106940 T503 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 106940 T503 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 106945 T527 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1
   [junit4]   2> 106945 T527 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/'
   [junit4]   2> 106987 T527 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
   [junit4]   2> 107028 T527 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 107029 T527 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 107034 T527 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 107052 T527 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 107053 T527 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 107053 T527 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/
   [junit4]   2> 107054 T527 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 107055 T527 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data
   [junit4]   2> 107056 T527 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index/
   [junit4]   2> 107056 T527 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 107056 T527 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index
   [junit4]   2> 107057 T527 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1531038594, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 107058 T527 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14eea4dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e229394),segFN=segments_1,generation=1}
   [junit4]   2> 107058 T527 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 107059 T527 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 107059 T527 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 107059 T527 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 107059 T527 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 107059 T527 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 107060 T527 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 107060 T527 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 107067 T527 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 107070 T527 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 107070 T527 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 107070 T527 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@36dd768d
   [junit4]   2> 107071 T527 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14eea4dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e229394),segFN=segments_1,generation=1}
   [junit4]   2> 107071 T527 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 107072 T527 oass.SolrIndexSearcher.<init> Opening Searcher@6e6cf504 main
   [junit4]   2> 107183 T527 oash.SnapPuller.startExecutorService Poll Scheduled at an interval of 1000ms
   [junit4]   2> 107184 T528 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6e6cf504 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 107184 T527 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 107185 T503 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
   [junit4]   2> 107185 T503 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C104 name=collection1 org.apache.solr.core.SolrCore@c660ebc
   [junit4]   2> 108929 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=91 
   [junit4]   2> 109030 T509 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={sort=id+desc&q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=191 
   [junit4]   2> 109231 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
   [junit4]   2> 109238 T509 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details&_trace=getDetails&wt=javabin&qt=/replication&version=2} status=0 QTime=7 
   [junit4]   2> 109260 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={slave=false&command=details&wt=javabin&qt=/replication&version=2} status=0 QTime=0 
   [junit4]   2> ASYNC  NEW_CORE C105 name=collection1 org.apache.solr.core.SolrCore@674e92d6
   [junit4]   2> 109268 T520 C105 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details&_trace=getDetails&wt=javabin&qt=/replication&version=2} status=0 QTime=13 
   [junit4]   2> 109418 T509 C104 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 45
   [junit4]   2> 109455 T509 C104 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 109458 T509 C104 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_2,generation=2}
   [junit4]   2> 109463 T509 C104 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 109465 T509 C104 oass.SolrIndexSearcher.<init> Opening Searcher@7711a38f main
   [junit4]   2> 109466 T509 C104 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 109466 T516 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7711a38f main{StandardDirectoryReader(segments_2:3:nrt _0(4.6):C1)}
   [junit4]   2> 109468 T509 C104 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 14
   [junit4]   2> 109484 T531 oash.SnapPuller.<init> WARN 'masterUrl' must be specified without the /replication suffix
   [junit4]   2> 109485 T531 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
   [junit4]   2> 109489 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
   [junit4]   2> 109490 T531 oash.SnapPuller.fetchLatestIndex Master's generation: 2
   [junit4]   2> 109491 T531 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
   [junit4]   2> 109491 T531 oash.SnapPuller.fetchLatestIndex Starting replication process
   [junit4]   2> 109503 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=8 
   [junit4]   2> 109505 T531 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 10
   [junit4]   2> 109507 T531 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051112809
   [junit4]   2> 109508 T531 oash.SnapPuller.fetchLatestIndex Starting download to MockDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/index3729637299tmp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@30b3608c) fullCopy=false
   [junit4]   2> 109525 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_0.fnm&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=6 
   [junit4]   2> 109549 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_0_Lucene41_0.doc&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
   [junit4]   2> 109557 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_0_nrm.cfe&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
   [junit4]   2> 109562 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
   [junit4]   2> 109569 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_0.fdx&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
   [junit4]   2> 109575 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_0_nrm.cfs&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
   [junit4]   2> 109580 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_0.si&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
   [junit4]   2> 109585 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_0_Lucene41_0.tim&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
   [junit4]   2> 109590 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_0_Lucene41_0.tip&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
   [junit4]   2> 109596 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_0.fdt&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
   [junit4]   2> 109597 T531 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
   [junit4]   2> 109599 T531 oasu.DefaultSolrCoreState.closeIndexWriter Closing IndexWriter...
   [junit4]   2> 109600 T531 oasu.DefaultSolrCoreState.closeIndexWriter Waiting until IndexWriter is unused... core=collection1
   [junit4]   2> 109600 T531 oasu.DefaultSolrCoreState.closeIndexWriter Rollback old IndexWriter... core=collection1
   [junit4]   2> 109615 T531 oasu.DefaultSolrCoreState.openIndexWriter Creating new IndexWriter...
   [junit4]   2> 109616 T531 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.1396484375, noCFSRatio=0.39545671228745916]
   [junit4]   2> 109625 T531 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14eea4dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e229394),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14eea4dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e229394),segFN=segments_2,generation=2}
   [junit4]   2> 109625 T531 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 109626 T531 oasu.DefaultSolrCoreState.openIndexWriter New IndexWriter is ready to be used.
   [junit4]   2> 109629 T531 oass.SolrIndexSearcher.<init> Opening Searcher@26e0108a main
   [junit4]   2> 109629 T528 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@26e0108a main{StandardDirectoryReader(segments_2:3:nrt _0(4.6):C1)}
   [junit4]   2> 109629 T531 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051112809 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051112809;done=true>>]
   [junit4]   2> 109630 T531 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051112809
   [junit4]   2> 109630 T521 C105 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={masterUrl=http://127.0.0.1:39548/solr/replication&command=fetchindex&wait=true} status=0 QTime=147 
   [junit4]   2> 109644 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={slave=false&command=details&wt=javabin&qt=/replication&version=2} status=0 QTime=1 
   [junit4]   2> 109645 T520 C105 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details&_trace=getDetails&wt=javabin&qt=/replication&version=2} status=0 QTime=7 
   [junit4]   2> 109659 T509 C104 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 8
   [junit4]   2> 109664 T509 C104 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 109666 T509 C104 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_3,generation=3}
   [junit4]   2> 109667 T509 C104 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 109668 T509 C104 oass.SolrIndexSearcher.<init> Opening Searcher@7dd84118 main
   [junit4]   2> 109669 T509 C104 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 109669 T516 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7dd84118 main{StandardDirectoryReader(segments_3:5:nrt _0(4.6):C1 _1(4.6):C1)}
   [junit4]   2> 109669 T509 C104 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 7
   [junit4]   2> 109672 T534 oash.SnapPuller.<init> WARN 'masterUrl' must be specified without the /replication suffix
   [junit4]   2> 109672 T534 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
   [junit4]   2> 109675 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
   [junit4]   2> 109676 T534 oash.SnapPuller.fetchLatestIndex Master's generation: 3
   [junit4]   2> 109676 T534 oash.SnapPuller.fetchLatestIndex Slave's generation: 2
   [junit4]   2> 109676 T534 oash.SnapPuller.fetchLatestIndex Starting replication process
   [junit4]   2> 109680 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=2 
   [junit4]   2> 109681 T534 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 19
   [junit4]   2> 109683 T534 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051112985
   [junit4]   2> 109683 T534 oash.SnapPuller.fetchLatestIndex Starting download to MockDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/index3729637300tmp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3a7f5639) fullCopy=false
   [junit4]   2> 109688 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_1_nrm.cfs&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=1 
   [junit4]   2> 109693 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_1_Lucene41_0.doc&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=1 
   [junit4]   2> 109694 T534 oash.SnapPuller.downloadIndexFiles Skipping download for _0.fnm because it already exists
   [junit4]   2> 109698 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_1_Lucene41_0.tim&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
   [junit4]   2> 109704 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_1_Lucene41_0.tip&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
   [junit4]   2> 109705 T534 oash.SnapPuller.downloadIndexFiles Skipping download for _0_nrm.cfs because it already exists
   [junit4]   2> 109710 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_1.fnm&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=1 
   [junit4]   2> 109715 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_1_nrm.cfe&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=1 
   [junit4]   2> 109721 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_1.fdx&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
   [junit4]   2> 109722 T534 oash.SnapPuller.downloadIndexFiles Skipping download for _0_Lucene41_0.doc because it already exists
   [junit4]   2> 109727 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_1.fdt&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
   [junit4]   2> 109733 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_1.si&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=1 
   [junit4]   2> 109734 T534 oash.SnapPuller.downloadIndexFiles Skipping download for _0_nrm.cfe because it already exists
   [junit4]   2> 109734 T534 oash.SnapPuller.downloadIndexFiles Skipping download for _0.fdx because it already exists
   [junit4]   2> 109735 T534 oash.SnapPuller.downloadIndexFiles Skipping download for _0.si because it already exists
   [junit4]   2> 109735 T534 oash.SnapPuller.downloadIndexFiles Skipping download for _0_Lucene41_0.tim because it already exists
   [junit4]   2> 109735 T534 oash.SnapPuller.downloadIndexFiles Skipping download for _0.fdt because it already exists
   [junit4]   2> 109735 T534 oash.SnapPuller.downloadIndexFiles Skipping download for _0_Lucene41_0.tip because it already exists
   [junit4]   2> 109739 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
   [junit4]   2> 109740 T534 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
   [junit4]   2> 109742 T534 oasu.DefaultSolrCoreState.closeIndexWriter Closing IndexWriter...
   [junit4]   2> 109742 T534 oasu.DefaultSolrCoreState.closeIndexWriter Waiting until IndexWriter is unused... core=collection1
   [junit4]   2> 109742 T534 oasu.DefaultSolrCoreState.closeIndexWriter Rollback old IndexWriter... core=collection1
   [junit4]   2> 109749 T534 oasu.DefaultSolrCoreState.openIndexWriter Creating new IndexWriter...
   [junit4]   2> 109750 T534 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.1396484375, noCFSRatio=0.39545671228745916]
   [junit4]   2> 109752 T534 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14eea4dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e229394),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14eea4dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e229394),segFN=segments_3,generation=3}
   [junit4]   2> 109753 T534 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 109753 T534 oasu.DefaultSolrCoreState.openIndexWriter New IndexWriter is ready to be used.
   [junit4]   2> 109757 T534 oass.SolrIndexSearcher.<init> Opening Searcher@6eca1272 main
   [junit4]   2> 109758 T528 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6eca1272 main{StandardDirectoryReader(segments_3:5:nrt _0(4.6):C1 _1(4.6):C1)}
   [junit4]   2> 109758 T534 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051112985 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051112985;done=true>>]
   [junit4]   2> 109759 T534 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051112985
   [junit4]   2> 109759 T521 C105 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={masterUrl=http://127.0.0.1:39548/solr/replication&command=fetchindex&wait=true} status=0 QTime=88 
   [junit4]   2> 109767 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={slave=false&command=details&wt=javabin&qt=/replication&version=2} status=0 QTime=0 
   [junit4]   2> 109769 T520 C105 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details&_trace=getDetails&wt=javabin&qt=/replication&version=2} status=0 QTime=6 
   [junit4]   2> 109783 T509 C104 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 8
   [junit4]   2> 109788 T509 C104 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 109790 T509 C104 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_3,generation=3}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f6cf858 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@693c4b81),segFN=segments_4,generation=4}
   [junit4]   2> 109791 T509 C104 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
   [junit4]   2> 109792 T509 C104 oass.SolrIndexSearcher.<init> Opening Searcher@1856310a main
   [junit4]   2> 109793 T509 C104 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 109793 T516 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1856310a main{StandardDirectoryReader(segments_4:7:nrt _0(4.6):C1 _1(4.6):C1 _2(4.6):C1)}
   [junit4]   2> 109794 T509 C104 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 7
   [junit4]   2> 109796 T536 oash.SnapPuller.<init> WARN 'masterUrl' must be specified without the /replication suffix
   [junit4]   2> 109796 T536 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
   [junit4]   2> 109799 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
   [junit4]   2> 109800 T536 oash.SnapPuller.fetchLatestIndex Master's generation: 4
   [junit4]   2> 109800 T536 oash.SnapPuller.fetchLatestIndex Slave's generation: 3
   [junit4]   2> 109800 T536 oash.SnapPuller.fetchLatestIndex Starting replication process
   [junit4]   2> 109805 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=2 
   [junit4]   2> 109806 T536 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 28
   [junit4]   2> 109808 T536 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051113110
   [junit4]   2> 109809 T536 oash.SnapPuller.fetchLatestIndex Starting download to MockDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/index3729637301tmp lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@10b44b04) fullCopy=false
   [junit4]   2> 109814 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_2.si&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
   [junit4]   2> 109815 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _1_Lucene41_0.doc because it already exists
   [junit4]   2> 109820 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_2_Lucene41_0.tim&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=1 
   [junit4]   2> 109826 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_2_Lucene41_0.tip&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=1 
   [junit4]   2> 109827 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _1.fnm because it already exists
   [junit4]   2> 109831 T508 C104 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_2_nrm.cfe&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=1 
   [junit4]   2> ASYNC  NEW_CORE C106 name=collection1 org.apache.solr.core.SolrCore@c660ebc
   [junit4]   2> 109845 T508 C106 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_2_nrm.cfs&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
   [junit4]   2> 109846 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _0.si because it already exists
   [junit4]   2> 109847 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _0_Lucene41_0.tim because it already exists
   [junit4]   2> 109847 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _0_Lucene41_0.tip because it already exists
   [junit4]   2> 109847 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _1_nrm.cfs because it already exists
   [junit4]   2> 109847 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _0.fnm because it already exists
   [junit4]   2> 109848 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _1_Lucene41_0.tim because it already exists
   [junit4]   2> 109852 T508 C106 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_2.fdt&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
   [junit4]   2> 109853 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _1_Lucene41_0.tip because it already exists
   [junit4]   2> 109857 T508 C106 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_2_Lucene41_0.doc&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
   [junit4]   2> 109858 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _0_nrm.cfs because it already exists
   [junit4]   2> 109862 T508 C106 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_2.fdx&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=1 
   [junit4]   2> 109863 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _1_nrm.cfe because it already exists
   [junit4]   2> 109867 T508 C106 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=_2.fnm&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
   [junit4]   2> 109868 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _1.fdx because it already exists
   [junit4]   2> 109869 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _1.fdt because it already exists
   [junit4]   2> 109869 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _0_Lucene41_0.doc because it already exists
   [junit4]   2> 109869 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _1.si because it already exists
   [junit4]   2> 109869 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _0_nrm.cfe because it already exists
   [junit4]   2> 109869 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _0.fdx because it already exists
   [junit4]   2> 109870 T536 oash.SnapPuller.downloadIndexFiles Skipping download for _0.fdt because it already exists
   [junit4]   2> 109874 T508 C106 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=1 
   [junit4]   2> 109875 T536 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
   [junit4]   2> 109876 T536 oasu.DefaultSolrCoreState.closeIndexWriter Closing IndexWriter...
   [junit4]   2> 109877 T536 oasu.DefaultSolrCoreState.closeIndexWriter Waiting until IndexWriter is unused... core=collection1
   [junit4]   2> 109877 T536 oasu.DefaultSolrCoreState.closeIndexWriter Rollback old IndexWriter... core=collection1
   [junit4]   2> 109883 T536 oasu.DefaultSolrCoreState.openIndexWriter Creating new IndexWriter...
   [junit4]   2> 109884 T536 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.1396484375, noCFSRatio=0.39545671228745916]
   [junit4]   2> 109887 T536 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14eea4dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e229394),segFN=segments_3,generation=3}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14eea4dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7e229394),segFN=segments_4,generation=4}
   [junit4]   2> 109888 T536 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
   [junit4]   2> 109888 T536 oasu.DefaultSolrCoreState.openIndexWriter New IndexWriter is ready to be used.
   [junit4]   2> 109893 T536 oass.SolrIndexSearcher.<init> Opening Searcher@6d541df4 main
   [junit4]   2> 109894 T528 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d541df4 main{StandardDirectoryReader(segments_4:7:nrt _0(4.6):C1 _1(4.6):C1 _2(4.6):C1)}
   [junit4]   2> 109894 T536 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051113110 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051113110;done=true>>]
   [junit4]   2> 109894 T536 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index.20131022051113110
   [junit4]   2> ASYNC  NEW_CORE C107 name=collection1 org.apache.solr.core.SolrCore@674e92d6
   [junit4]   2> 109895 T521 C107 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={masterUrl=http://127.0.0.1:39548/solr/replication&command=fetchindex&wait=true} status=0 QTime=100 
   [junit4]   2> 109916 T503 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 109919 T503 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:37064
   [junit4]   2> 109919 T503 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 109920 T503 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 109920 T503 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater
   [junit4]   2> 109921 T503 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/'
   [junit4]   2> 109963 T503 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/solr.xml
   [junit4]   2> 110057 T503 oasc.CoreContainer.<init> New CoreContainer 456681058
   [junit4]   2> 110057 T503 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/]
   [junit4]   2> 110059 T503 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 110060 T503 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 110060 T503 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 110061 T503 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 110061 T503 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 110062 T503 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 110062 T503 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 110062 T503 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 110063 T503 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 110070 T503 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 110070 T503 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 110078 T547 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1
   [junit4]   2> 110078 T547 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/'
   [junit4]   2> 110145 T547 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
   [junit4]   2> 110214 T547 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 110215 T547 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 110223 T547 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 110251 T547 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 110253 T547 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 110254 T547 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data/
   [junit4]   2> 110254 T547 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 110256 T547 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data
   [junit4]   2> 110257 T547 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data/index/
   [junit4]   2> 110257 T547 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 110258 T547 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data/index
   [junit4]   2> 110258 T547 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1531038594, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 110259 T547 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3643614d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5b889cda),segFN=segments_1,generation=1}
   [junit4]   2> 110260 T547 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 110261 T547 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 110261 T547 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 110261 T547 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 110262 T547 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 110262 T547 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 110262 T547 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 110262 T547 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 110274 T547 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 110278 T547 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 110278 T547 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 110279 T547 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@464274bf
   [junit4]   2> 110280 T547 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3643614d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5b889cda),segFN=segments_1,generation=1}
   [junit4]   2> 110280 T547 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 110280 T547 oass.SolrIndexSearcher.<init> Opening Searcher@69316a2e main
   [junit4]   2> 110281 T547 oash.SnapPuller.<init> WARN 'masterUrl' must be specified without the /replication suffix
   [junit4]   2> 110281 T547 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
   [junit4]   2> 110281 T547 oash.ReplicationHandler.inform Replication enabled for following config files: schema.xml
   [junit4]   2> 110282 T547 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 110282 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@69316a2e main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 110282 T547 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 110283 T503 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
   [junit4]   2> 110284 T503 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 110301 T508 C106 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={slave=false&command=details&wt=javabin&qt=/replication&version=2} status=0 QTime=0 
   [junit4]   2> ASYNC  NEW_CORE C108 name=collection1 org.apache.solr.core.SolrCore@3369d7a
   [junit4]   2> 110302 T540 C108 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details&_trace=getDetails&wt=javabin&qt=/replication&version=2} status=0 QTime=5 
   [junit4]   2> 110305 T503 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=456681058
   [junit4]   2> 110306 T503 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3369d7a
   [junit4]   2> 110307 T503 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 110307 T503 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 110308 T503 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 110308 T503 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 110309 T503 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 110310 T503 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 110310 T503 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data/index;done=false>>]
   [junit4]   2> 110311 T503 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data/index
   [junit4]   2> 110311 T503 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data;done=false>>]
   [junit4]   2> 110312 T503 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473199/repeater/collection1/data
   [junit4]   2> 110313 T503 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 110377 T503 oas.SolrTestCaseJ4.tearDown ###Ending doTestDetails
   [junit4]   2> 110378 T503 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=2078185786
   [junit4]   2> 110379 T503 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@c660ebc
   [junit4]   2> 110380 T503 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=3,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=3,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 110381 T503 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 110381 T503 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 110382 T503 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 110383 T503 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 110384 T503 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 110384 T503 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data/index;done=false>>]
   [junit4]   2> 110385 T503 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data/index
   [junit4]   2> 110385 T503 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data;done=false>>]
   [junit4]   2> 110386 T503 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393468131/master/collection1/data
   [junit4]   2> 110386 T503 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 110440 T503 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1950058271
   [junit4]   2> 110441 T503 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@674e92d6
   [junit4]   2> 110442 T503 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 110443 T503 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 110443 T503 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 110444 T503 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 110445 T503 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 110445 T503 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 110446 T503 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data;done=false>>]
   [junit4]   2> 110446 T503 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data
   [junit4]   2> 110447 T503 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index;done=false>>]
   [junit4]   2> 110447 T503 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393470130/slave/collection1/data/index
   [junit4]   2> 110448 T503 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 110519 T503 oas.SolrTestCaseJ4.setUp ###Starting doTestReplicateAfterStartupWithNoActivity
   [junit4]   2> 110538 T503 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 110541 T503 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63827
   [junit4]   2> 110542 T503 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 110542 T503 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 110542 T503 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473823/master
   [junit4]   2> 110543 T503 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473823/master/'
   [junit4]   2> 110581 T503 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473823/master/solr.xml
   [junit4]   2> 110674 T503 oasc.CoreContainer.<init> New CoreContainer 2029831523
   [junit4]   2> 110674 T503 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473823/master/]
   [junit4]   2> 110676 T503 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 110677 T503 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 110677 T503 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 110678 T503 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 110678 T503 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 110679 T503 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 110679 T503 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 110679 T503 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 110680 T503 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 110689 T503 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 110689 T503 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 110697 T559 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473823/master/collection1
   [junit4]   2> 110697 T559 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473823/master/collection1/'
   [junit4]   2> 110772 T559 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
   [junit4]   2> 110846 T559 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 110847 T559 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 110856 T559 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 110886 T559 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 110888 T559 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 110889 T559 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473823/master/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1382393473823/master/collection1/data/
   [junit4]   2> 110889 T559 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 110891 T559 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.ha

[...truncated too long message...]

olr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:156)
   [junit4]   2> 	at org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:118)
   [junit4]   2> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:679)
   [junit4]   2> Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to 127.0.0.1:62905 timed out
   [junit4]   2> 	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:129)
   [junit4]   2> 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
   [junit4]   2> 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
   [junit4]   2> 	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643)
   [junit4]   2> 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
   [junit4]   2> 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
   [junit4]   2> 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
   [junit4]   2> 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
   [junit4]   2> 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:366)
   [junit4]   2> 	... 11 more
   [junit4]   2> Caused by: java.net.SocketTimeoutException: connect timed out
   [junit4]   2> 	at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]   2> 	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
   [junit4]   2> 	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
   [junit4]   2> 	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
   [junit4]   2> 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:385)
   [junit4]   2> 	at java.net.Socket.connect(Socket.java:546)
   [junit4]   2> 	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
   [junit4]   2> 	... 19 more
   [junit4]   2> 
   [junit4]   2> 1728827 T3632 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55332/k/j DONE. sync failed
   [junit4]   2> 1728828 T3632 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
   [junit4]   2> 1728828 T3632 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
   [junit4]   2> 1728828 T3632 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@55c41469
   [junit4]   2> 1728830 T3632 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=10,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=305,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 1728830 T3632 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 1728831 T3632 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1728831 T3632 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1728832 T3632 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 1728832 T3632 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1728833 T3632 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.SyncSliceTest-1382394892632/jetty6/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.SyncSliceTest-1382394892632/jetty6/index;done=false>>]
   [junit4]   2> 1728833 T3632 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.SyncSliceTest-1382394892632/jetty6/index
   [junit4]   2> 1728833 T3632 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.SyncSliceTest-1382394892632/jetty6 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.SyncSliceTest-1382394892632/jetty6;done=false>>]
   [junit4]   2> 1728833 T3632 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.SyncSliceTest-1382394892632/jetty6
   [junit4]   2> 1728834 T3632 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:247)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:244)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:244)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1728834 T3632 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_OM, timezone=Pacific/Wallis
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=3,free=150707816,total=371720192
   [junit4]   2> NOTE: All tests run in this JVM: [TestSearchPerf, TestQueryUtils, TestRealTimeGet, OverseerTest, TestReplicationHandler, TestSystemIdResolver, QueryEqualityTest, AutoCommitTest, TestFaceting, UUIDFieldTest, TestNRTOpen, TestNoOpRegenerator, BasicFunctionalityTest, TestCharFilters, SolrIndexSplitterTest, TestDynamicFieldCollectionResource, DistributedTermsComponentTest, HdfsDirectoryTest, LoggingHandlerTest, DistributedSpellCheckComponentTest, DOMUtilTest, TestGroupingSearch, EchoParamsTest, FieldMutatingUpdateProcessorTest, TestSort, SpatialFilterTest, OverseerCollectionProcessorTest, TestPostingsSolrHighlighter, InfoHandlerTest, HdfsUnloadDistributedZkTest, LeaderElectionTest, WordBreakSolrSpellCheckerTest, TestCloudManagedSchemaAddField, SynonymTokenizerTest, CoreAdminCreateDiscoverTest, TestSolrXmlPersistor, TestSolrXml, TestSolrIndexConfig, MoreLikeThisHandlerTest, ClusterStateUpdateTest, IndexSchemaTest, SampleTest, TestFunctionQuery, TestBM25SimilarityFactory, MBeansHandlerTest, SuggesterFSTTest, ZkControllerTest, TestSuggestSpellingConverter, TestSchemaResource, FastVectorHighlighterTest, TestUniqueKeyFieldResource, TestSchemaNameResource, BinaryUpdateRequestHandlerTest, RegexBoostProcessorTest, CoreAdminHandlerTest, SearchHandlerTest, TestSolrQueryParserDefaultOperatorResource, DocValuesTest, TestLuceneMatchVersion, TermsComponentTest, TestDocumentBuilder, StatelessScriptUpdateProcessorFactoryTest, DocValuesMissingTest, ZkCLITest, QueryParsingTest, ExternalFileFieldSortTest, TestShardHandlerFactory, TestReversedWildcardFilterFactory, SolrTestCaseJ4Test, TestBadConfig, TestExtendedDismaxParser, PeerSyncTest, FileBasedSpellCheckerTest, ResourceLoaderTest, TestLRUCache, HdfsBasicDistributedZk2Test, DocumentBuilderTest, TestDocSet, SOLR749Test, BasicDistributedZkTest, UnloadDistributedZkTest, SyncSliceTest]
   [junit4] Completed on J0 in 199.91s, 1 test, 1 failure <<< FAILURES!

[...truncated 514 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:435: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:408: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/common-build.xml:483: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1263: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:906: There were test failures: 327 suites, 1453 tests, 2 failures, 33 ignored (4 assumptions)

Total time: 131 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 414 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/414/

1 tests failed.
FAILED:  org.apache.lucene.search.suggest.DocumentExpressionDictionaryTest.testWithoutPayload

Error Message:
CompositeReader is not supported

Stack Trace:
java.lang.IllegalArgumentException: CompositeReader is not supported
	at __randomizedtesting.SeedInfo.seed([4393102586925672:F90997B172D8B897]:0)
	at org.apache.lucene.search.suggest.DocumentExpressionDictionary$DocumentExpressionInputIterator.<init>(DocumentExpressionDictionary.java:110)
	at org.apache.lucene.search.suggest.DocumentExpressionDictionary.getWordsIterator(DocumentExpressionDictionary.java:98)
	at org.apache.lucene.search.suggest.DocumentExpressionDictionaryTest.testWithoutPayload(DocumentExpressionDictionaryTest.java:128)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:679)




Build Log:
[...truncated 8359 lines...]
   [junit4] Suite: org.apache.lucene.search.suggest.DocumentExpressionDictionaryTest
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=DocumentExpressionDictionaryTest -Dtests.method=testWithoutPayload -Dtests.seed=4393102586925672 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=zh_HK -Dtests.timezone=Pacific/Rarotonga -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.21s J1 | DocumentExpressionDictionaryTest.testWithoutPayload <<<
   [junit4]    > Throwable #1: java.lang.IllegalArgumentException: CompositeReader is not supported
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4393102586925672:F90997B172D8B897]:0)
   [junit4]    > 	at org.apache.lucene.search.suggest.DocumentExpressionDictionary$DocumentExpressionInputIterator.<init>(DocumentExpressionDictionary.java:110)
   [junit4]    > 	at org.apache.lucene.search.suggest.DocumentExpressionDictionary.getWordsIterator(DocumentExpressionDictionary.java:98)
   [junit4]    > 	at org.apache.lucene.search.suggest.DocumentExpressionDictionaryTest.testWithoutPayload(DocumentExpressionDictionaryTest.java:128)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:679)
   [junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=183), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=183)), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {f1=DFR GB3(800.0)}, locale=zh_HK, timezone=Pacific/Rarotonga
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=1,free=447858464,total=514129920
   [junit4]   2> NOTE: All tests run in this JVM: [TestInputIterator, WFSTCompletionTest, TestWordBreakSpellChecker, DocumentExpressionDictionaryTest]
   [junit4] Completed on J1 in 0.79s, 3 tests, 1 error <<< FAILURES!

[...truncated 49 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:435: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:408: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/build.xml:562: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1919: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/module-build.xml:60: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1263: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:906: There were test failures: 24 suites, 128 tests, 1 error, 1 ignored

Total time: 68 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 413 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/413/

1 tests failed.
REGRESSION:  org.apache.lucene.index.Test2BPostings.test

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([984644A180EB9197:10127B7B2E17FC6F]:0)
	at org.apache.lucene.util.ByteBlockPool$DirectTrackingAllocator.getByteBlock(ByteBlockPool.java:103)
	at org.apache.lucene.util.ByteBlockPool.nextBuffer(ByteBlockPool.java:202)
	at org.apache.lucene.util.ByteBlockPool.allocSlice(ByteBlockPool.java:257)
	at org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:252)
	at org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:273)
	at org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:204)
	at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:235)
	at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:165)
	at org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:248)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:254)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:446)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1532)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1202)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1183)
	at org.apache.lucene.index.Test2BPostings.test(Test2BPostings.java:76)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)




Build Log:
[...truncated 438 lines...]
   [junit4] Suite: org.apache.lucene.index.Test2BPostings
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=Test2BPostings -Dtests.method=test -Dtests.seed=984644A180EB9197 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=sl_SI -Dtests.timezone=Europe/Vaduz -Dtests.file.encoding=UTF-8
   [junit4] ERROR    271s J0 | Test2BPostings.test <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([984644A180EB9197:10127B7B2E17FC6F]:0)
   [junit4]    > 	at org.apache.lucene.util.ByteBlockPool$DirectTrackingAllocator.getByteBlock(ByteBlockPool.java:103)
   [junit4]    > 	at org.apache.lucene.util.ByteBlockPool.nextBuffer(ByteBlockPool.java:202)
   [junit4]    > 	at org.apache.lucene.util.ByteBlockPool.allocSlice(ByteBlockPool.java:257)
   [junit4]    > 	at org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:252)
   [junit4]    > 	at org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:273)
   [junit4]    > 	at org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:204)
   [junit4]    > 	at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:235)
   [junit4]    > 	at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:165)
   [junit4]    > 	at org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:248)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:254)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:446)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1532)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1202)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1183)
   [junit4]    > 	at org.apache.lucene.index.Test2BPostings.test(Test2BPostings.java:76)
   [junit4]   2> NOTE: test params are: codec=Lucene45: {field=MockVariableIntBlock(baseBlockSize=96)}, docValues:{}, sim=DefaultSimilarity, locale=sl_SI, timezone=Europe/Vaduz
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=1,free=152528120,total=477233152
   [junit4]   2> NOTE: All tests run in this JVM: [TestSimilarity, TestSimpleAttributeImpl, TestRegexpQuery, TestDocValuesWithThreads, TestPostingsFormat, TestNumericRangeQuery32, TestCharTermAttributeImpl, TestIndexableBinaryStringTools, TestDirectoryReader, TestFastDecompressionMode, TestCompressingTermVectorsFormat, TestBooleanMinShouldMatch, TestEliasFanoDocIdSet, TestParallelTermEnum, TestDocumentsWriterStallControl, TestFieldCacheRangeFilter, Test2BPostings]
   [junit4] Completed on J0 in 271.41s, 1 test, 1 error <<< FAILURES!

[...truncated 1046 lines...]
   [junit4] JVM J0: stdout was not empty, see: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/build/core/test/temp/junit4-J0-20131022_233611_993.sysout
   [junit4] >>> JVM J0: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/heapdumps/java_pid46487.hprof ...
   [junit4] Heap dump file created [424921456 bytes in 0.802 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 9 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:435: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:408: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/build.xml:49: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1263: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:906: There were test failures: 378 suites, 2709 tests, 1 error, 68 ignored (55 assumptions)

Total time: 34 minutes 11 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure