You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by ma...@apache.org on 2024/01/08 17:57:46 UTC
(tomcat) branch 9.0.x updated (7d3a116daf -> 02e01b80f3)
This is an automated email from the ASF dual-hosted git repository.
markt pushed a change to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
from 7d3a116daf Cleanup connector names in json
new dcad392264 Refactor absolute performance tests so they do not run by default
new 4627204ed2 Add additional comment to existing absolute performance tests
new aebc402308 Refactor so these relative performance tests generate pass/fail results
new 4332382bb1 Add additional comment to this relative performance test
new 5d04b80899 Comment this absolute performance test with an upper limit
new 08d7da2c39 This test is very timing sensitive. Add some test specific margins.
new f5213adc30 Update the docs for running the tests
new 02e01b80f3 Don't test lazy vs full as they can be quite close
The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
BUILDING.txt | 17 ++++++--
...a => TesterCompositeELResolverPerformance.java} | 6 ++-
test/javax/el/TesterImportHandlerPerformance.java | 4 ++
...esterScopedAttributeELResolverPerformance.java} | 6 ++-
.../TesterContainerProviderPerformance.java | 4 ++
.../TesterDigestAuthenticatorPerformance.java | 4 ++
.../connector/TestResponsePerformance.java | 9 +++--
...formance.java => TesterRequestPerformance.java} | 6 ++-
...> TesterApplicationHttpRequestPerformance.java} | 6 ++-
...> TesterDefaultInstanceManagerPerformance.java} | 9 +++--
.../catalina/mapper/TestMapperPerformance.java | 3 ++
... TesterAbstractFileResourceSetPerformance.java} | 6 ++-
.../apache/el/parser/TestELParserPerformance.java | 10 +++++
...e.java => TesterTagHandlerPoolPerformance.java} | 7 +++-
.../juli/TestOneLineFormatterPerformance.java | 31 +++++++++-----
...> TesterOneLineFormatterMillisPerformance.java} | 6 ++-
.../tomcat/unittest/TesterThreadedPerformance.java | 3 ++
.../apache/tomcat/util/bcel/TesterPerformance.java | 6 +++
.../util/buf/TestCharsetCachePerformance.java | 31 +++++++-------
.../tomcat/util/http/TesterCookiesPerformance.java | 4 ++
...va => TesterFastHttpDateFormatPerformance.java} | 6 ++-
.../util/http/TesterParametersPerformance.java | 4 ++
....java => TesterConnectionLimitPerformance.java} | 6 ++-
.../server/TestAsyncMessagesPerformance.java | 47 +++++++++++++++++-----
24 files changed, 187 insertions(+), 54 deletions(-)
rename test/javax/el/{TestCompositeELResolverPerformance.java => TesterCompositeELResolverPerformance.java} (88%)
rename test/javax/servlet/jsp/el/{TestScopedAttributeELResolverPerformance.java => TesterScopedAttributeELResolverPerformance.java} (87%)
rename test/org/apache/catalina/connector/{TestRequestPerformance.java => TesterRequestPerformance.java} (87%)
rename test/org/apache/catalina/core/{TestApplicationHttpRequestPerformance.java => TesterApplicationHttpRequestPerformance.java} (88%)
rename test/org/apache/catalina/core/{TestDefaultInstanceManagerPerformance.java => TesterDefaultInstanceManagerPerformance.java} (92%)
rename test/org/apache/catalina/webresources/{TestAbstractFileResourceSetPerformance.java => TesterAbstractFileResourceSetPerformance.java} (93%)
rename test/org/apache/jasper/runtime/{TestTagHandlerPoolPerformance.java => TesterTagHandlerPoolPerformance.java} (92%)
rename test/org/apache/juli/{TestOneLineFormatterMillisPerformance.java => TesterOneLineFormatterMillisPerformance.java} (92%)
rename test/org/apache/tomcat/util/http/{TestFastHttpDateFormatPerformance.java => TesterFastHttpDateFormatPerformance.java} (89%)
rename test/org/apache/tomcat/websocket/{TestConnectionLimitPerformance.java => TesterConnectionLimitPerformance.java} (93%)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
(tomcat) 08/08: Don't test lazy vs full as they can be quite close
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 02e01b80f31380671d390e3739068df39e63128d
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Mon Jan 8 13:46:16 2024 +0000
Don't test lazy vs full as they can be quite close
---
test/org/apache/tomcat/util/buf/TestCharsetCachePerformance.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/test/org/apache/tomcat/util/buf/TestCharsetCachePerformance.java b/test/org/apache/tomcat/util/buf/TestCharsetCachePerformance.java
index f17098488e..0a0646db39 100644
--- a/test/org/apache/tomcat/util/buf/TestCharsetCachePerformance.java
+++ b/test/org/apache/tomcat/util/buf/TestCharsetCachePerformance.java
@@ -37,7 +37,8 @@ public class TestCharsetCachePerformance {
Assert.assertTrue("No cache was faster than full cache", timeFull < timeNone);
Assert.assertTrue("No cache was faster than lazy cache", timeLazy < timeNone);
- Assert.assertTrue("Lazy cache was faster than full cache ", timeFull < timeLazy);
+ // On average full cache is faster than lazy cache but they are close enough the test will fail sometimes
+ //Assert.assertTrue("Lazy cache was faster than full cache ", timeFull < timeLazy);
}
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
(tomcat) 06/08: This test is very timing sensitive. Add some test specific margins.
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 08d7da2c3998f7b2ee6c2e804d26274c7a725aa3
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Mon Jan 8 12:11:31 2024 +0000
This test is very timing sensitive. Add some test specific margins.
The current margins are based on local testing and reports on the dev
list. They may need to be relaxed further or if the results are very
inconsistent between systems it may need to be disabled by default.
---
.../server/TestAsyncMessagesPerformance.java | 47 +++++++++++++++++-----
1 file changed, 38 insertions(+), 9 deletions(-)
diff --git a/test/org/apache/tomcat/websocket/server/TestAsyncMessagesPerformance.java b/test/org/apache/tomcat/websocket/server/TestAsyncMessagesPerformance.java
index 40ace938ca..926073af0f 100644
--- a/test/org/apache/tomcat/websocket/server/TestAsyncMessagesPerformance.java
+++ b/test/org/apache/tomcat/websocket/server/TestAsyncMessagesPerformance.java
@@ -36,6 +36,11 @@ import org.apache.catalina.startup.TomcatBaseTest;
import org.apache.tomcat.websocket.TesterAsyncTiming;
import org.apache.tomcat.websocket.TesterMessageCountClient.TesterProgrammaticEndpoint;
+/*
+ * This test is very timing sensitive. Any failures need to be checked to see if the thresholds just need adjusting to
+ * support a wider range of platforms and/or Java versions or if the failure is an indication of a performance drop in
+ * the WebSocket implementation.
+ */
public class TestAsyncMessagesPerformance extends TomcatBaseTest {
@Test
@@ -69,11 +74,20 @@ public class TestAsyncMessagesPerformance extends TomcatBaseTest {
private long lastMessage = 0;
private int sequence = 0;
private int count = 0;
+ private long seqZeroTimingFailureCount = 0;
+ private long seqOneTimingFailureCount = 0;
+ private long seqTwoTimingFailureCount = 0;
+
private CountDownLatch latch = new CountDownLatch(1);
private volatile boolean fail = false;
@Override
public void onMessage(ByteBuffer message, boolean last) {
+ // Expected received data is:
+ // 1 * 16k message in 2 * 8k chunks
+ // 1 * 4k message in 1 * 4k chunk
+ // 50 ms pause
+ // loop
if (lastMessage == 0) {
// First message. Don't check
sequence++;
@@ -87,34 +101,36 @@ public class TestAsyncMessagesPerformance extends TomcatBaseTest {
sequence++;
if (message.capacity() != 8192) {
System.out.println(
- "Expected size 8192 but was [" + message.capacity() + "], count [" + count + "]");
+ "SEQ0: Expected size 8192 but was [" + message.capacity() + "], count [" + count + "]");
fail = true;
}
if (diff < 40000000) {
- System.out.println("Expected diff > 40ms but was [" + diff + "], count [" + count + "]");
- fail = true;
+ System.out.println("SEQ0: Expected diff > 40ms but was [" + diff + "], count [" + count + "]");
+ seqZeroTimingFailureCount++;
}
} else if (sequence == 1) {
sequence++;
if (message.capacity() != 8192) {
System.out.println(
- "Expected size 8192 but was [" + message.capacity() + "], count [" + count + "]");
+ "SEQ1: Expected size 8192 but was [" + message.capacity() + "], count [" + count + "]");
fail = true;
}
+ // Gap between 2* 8k chunks of 16k message expected to be less than 0.5ms
if (diff > 500000) {
- System.out.println("Expected diff < 500,000 but was [" + diff + "], count [" + count + "]");
- fail = true;
+ System.out.println("SEQ1: Expected diff < 500,000 but was [" + diff + "], count [" + count + "]");
+ seqOneTimingFailureCount++;
}
} else if (sequence == 2) {
sequence = 0;
if (message.capacity() != 4096) {
System.out.println(
- "Expected size 4096 but was [" + message.capacity() + "], count [" + count + "]");
+ "SEQ2: Expected size 4096 but was [" + message.capacity() + "], count [" + count + "]");
fail = true;
}
+ // Gap between 16k message and 4k message expected to be less than 0.5ms
if (diff > 500000) {
- System.out.println("Expected diff < 500,000 but was [" + diff + "], count [" + count + "]");
- fail = true;
+ System.out.println("SEQ2: Expected diff < 500,000 but was [" + diff + "], count [" + count + "]");
+ seqTwoTimingFailureCount++;
}
}
}
@@ -130,6 +146,19 @@ public class TestAsyncMessagesPerformance extends TomcatBaseTest {
}
public boolean hasFailed() {
+ // Total iterations are 1500
+ if (!fail) {
+ if (seqZeroTimingFailureCount > 1) {
+ // The 50ms pause after the short message may very rarely appear to be less than 40ms
+ fail = true;
+ } else if (seqOneTimingFailureCount > 5) {
+ // The two chunks of the 16k message may rarely be more than 0.5ms apart
+ fail = true;
+ } else if (seqTwoTimingFailureCount > 100) {
+ // The short message may often be more than 0.5ms after the long message
+ fail = true;
+ }
+ }
return fail;
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
(tomcat) 01/08: Refactor absolute performance tests so they do not run by default
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit dcad3922644dbb59cd30e0436ef9abbadb99bbf0
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Mon Jan 8 12:06:19 2024 +0000
Refactor absolute performance tests so they do not run by default
Absolute performance tests report the time taken to complete a series of
actions. They cannot fail therefore do not run them as part of a
standard test run.
These tests are excluded by naming them Tester...
---
...erformance.java => TesterCompositeELResolverPerformance.java} | 6 +++++-
...ance.java => TesterScopedAttributeELResolverPerformance.java} | 6 +++++-
...TestRequestPerformance.java => TesterRequestPerformance.java} | 6 +++++-
...ormance.java => TesterApplicationHttpRequestPerformance.java} | 6 +++++-
...ormance.java => TesterDefaultInstanceManagerPerformance.java} | 9 +++++----
...rmance.java => TesterAbstractFileResourceSetPerformance.java} | 6 +++++-
...PoolPerformance.java => TesterTagHandlerPoolPerformance.java} | 7 +++++--
...ormance.java => TesterOneLineFormatterMillisPerformance.java} | 6 +++++-
...Performance.java => TesterFastHttpDateFormatPerformance.java} | 6 +++++-
...mitPerformance.java => TesterConnectionLimitPerformance.java} | 6 +++++-
10 files changed, 50 insertions(+), 14 deletions(-)
diff --git a/test/javax/el/TestCompositeELResolverPerformance.java b/test/javax/el/TesterCompositeELResolverPerformance.java
similarity index 88%
rename from test/javax/el/TestCompositeELResolverPerformance.java
rename to test/javax/el/TesterCompositeELResolverPerformance.java
index b31907cab5..cd913c1d20 100644
--- a/test/javax/el/TestCompositeELResolverPerformance.java
+++ b/test/javax/el/TesterCompositeELResolverPerformance.java
@@ -18,7 +18,11 @@ package javax.el;
import org.junit.Test;
-public class TestCompositeELResolverPerformance {
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
+public class TesterCompositeELResolverPerformance {
/*
* https://bz.apache.org/bugzilla/show_bug.cgi?id=68119
diff --git a/test/javax/servlet/jsp/el/TestScopedAttributeELResolverPerformance.java b/test/javax/servlet/jsp/el/TesterScopedAttributeELResolverPerformance.java
similarity index 87%
rename from test/javax/servlet/jsp/el/TestScopedAttributeELResolverPerformance.java
rename to test/javax/servlet/jsp/el/TesterScopedAttributeELResolverPerformance.java
index 9295155bd4..adaeabd010 100644
--- a/test/javax/servlet/jsp/el/TestScopedAttributeELResolverPerformance.java
+++ b/test/javax/servlet/jsp/el/TesterScopedAttributeELResolverPerformance.java
@@ -25,7 +25,11 @@ import javax.servlet.jsp.TesterPageContext;
import org.junit.Test;
-public class TestScopedAttributeELResolverPerformance {
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
+public class TesterScopedAttributeELResolverPerformance {
/*
* With the caching of NotFound responses this test takes ~20ms. Without the
diff --git a/test/org/apache/catalina/connector/TestRequestPerformance.java b/test/org/apache/catalina/connector/TesterRequestPerformance.java
similarity index 87%
rename from test/org/apache/catalina/connector/TestRequestPerformance.java
rename to test/org/apache/catalina/connector/TesterRequestPerformance.java
index 81a00829ce..42ec455051 100644
--- a/test/org/apache/catalina/connector/TestRequestPerformance.java
+++ b/test/org/apache/catalina/connector/TesterRequestPerformance.java
@@ -20,7 +20,11 @@ import org.junit.Test;
import org.apache.tomcat.unittest.TesterRequest;
-public class TestRequestPerformance {
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
+public class TesterRequestPerformance {
@Test
public void localeParsePerformance() throws Exception {
diff --git a/test/org/apache/catalina/core/TestApplicationHttpRequestPerformance.java b/test/org/apache/catalina/core/TesterApplicationHttpRequestPerformance.java
similarity index 88%
rename from test/org/apache/catalina/core/TestApplicationHttpRequestPerformance.java
rename to test/org/apache/catalina/core/TesterApplicationHttpRequestPerformance.java
index 1592ceba2f..03d5932eb3 100644
--- a/test/org/apache/catalina/core/TestApplicationHttpRequestPerformance.java
+++ b/test/org/apache/catalina/core/TesterApplicationHttpRequestPerformance.java
@@ -20,7 +20,11 @@ import org.junit.Test;
import org.apache.catalina.connector.Request;
-public class TestApplicationHttpRequestPerformance {
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
+public class TesterApplicationHttpRequestPerformance {
@Test
public void testGetAttribute() {
diff --git a/test/org/apache/catalina/core/TestDefaultInstanceManagerPerformance.java b/test/org/apache/catalina/core/TesterDefaultInstanceManagerPerformance.java
similarity index 92%
rename from test/org/apache/catalina/core/TestDefaultInstanceManagerPerformance.java
rename to test/org/apache/catalina/core/TesterDefaultInstanceManagerPerformance.java
index c496efeb08..cbea78df9c 100644
--- a/test/org/apache/catalina/core/TestDefaultInstanceManagerPerformance.java
+++ b/test/org/apache/catalina/core/TesterDefaultInstanceManagerPerformance.java
@@ -28,11 +28,12 @@ import org.apache.catalina.startup.Tomcat;
import org.apache.catalina.startup.TomcatBaseTest;
import org.apache.tomcat.InstanceManager;
-public class TestDefaultInstanceManagerPerformance extends TomcatBaseTest {
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
+public class TesterDefaultInstanceManagerPerformance extends TomcatBaseTest {
- /*
- * Performance test. Comment out @Ignore to run the test.
- */
@Test
public void testConcurrency() throws Exception {
// Create a populated InstanceManager
diff --git a/test/org/apache/catalina/webresources/TestAbstractFileResourceSetPerformance.java b/test/org/apache/catalina/webresources/TesterAbstractFileResourceSetPerformance.java
similarity index 93%
rename from test/org/apache/catalina/webresources/TestAbstractFileResourceSetPerformance.java
rename to test/org/apache/catalina/webresources/TesterAbstractFileResourceSetPerformance.java
index 5b17d116a1..7d66c129be 100644
--- a/test/org/apache/catalina/webresources/TestAbstractFileResourceSetPerformance.java
+++ b/test/org/apache/catalina/webresources/TesterAbstractFileResourceSetPerformance.java
@@ -20,7 +20,11 @@ import java.util.regex.Pattern;
import org.junit.Test;
-public class TestAbstractFileResourceSetPerformance {
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
+public class TesterAbstractFileResourceSetPerformance {
private static final Pattern UNSAFE_WINDOWS_FILENAME_PATTERN = Pattern.compile(" $|[\"<>]");
diff --git a/test/org/apache/jasper/runtime/TestTagHandlerPoolPerformance.java b/test/org/apache/jasper/runtime/TesterTagHandlerPoolPerformance.java
similarity index 92%
rename from test/org/apache/jasper/runtime/TestTagHandlerPoolPerformance.java
rename to test/org/apache/jasper/runtime/TesterTagHandlerPoolPerformance.java
index 41dd444b36..a810ac2443 100644
--- a/test/org/apache/jasper/runtime/TestTagHandlerPoolPerformance.java
+++ b/test/org/apache/jasper/runtime/TesterTagHandlerPoolPerformance.java
@@ -30,8 +30,11 @@ import org.apache.catalina.startup.TomcatBaseTest;
import org.apache.tomcat.unittest.TesterThreadedPerformance;
import org.apache.tomcat.unittest.tags.Bug53545;
-
-public class TestTagHandlerPoolPerformance extends TomcatBaseTest {
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
+public class TesterTagHandlerPoolPerformance extends TomcatBaseTest {
@Test
public void testConcurrency() throws Exception {
diff --git a/test/org/apache/juli/TestOneLineFormatterMillisPerformance.java b/test/org/apache/juli/TesterOneLineFormatterMillisPerformance.java
similarity index 92%
rename from test/org/apache/juli/TestOneLineFormatterMillisPerformance.java
rename to test/org/apache/juli/TesterOneLineFormatterMillisPerformance.java
index b7c0cabd28..41dc3a8927 100644
--- a/test/org/apache/juli/TestOneLineFormatterMillisPerformance.java
+++ b/test/org/apache/juli/TesterOneLineFormatterMillisPerformance.java
@@ -25,8 +25,12 @@ import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameter;
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
@RunWith(Parameterized.class)
-public class TestOneLineFormatterMillisPerformance {
+public class TesterOneLineFormatterMillisPerformance {
@Parameterized.Parameters(name = "{index}: format[{0}]")
public static Collection<Object[]> parameters() {
diff --git a/test/org/apache/tomcat/util/http/TestFastHttpDateFormatPerformance.java b/test/org/apache/tomcat/util/http/TesterFastHttpDateFormatPerformance.java
similarity index 89%
rename from test/org/apache/tomcat/util/http/TestFastHttpDateFormatPerformance.java
rename to test/org/apache/tomcat/util/http/TesterFastHttpDateFormatPerformance.java
index ad3e176cfb..4d6e579e91 100644
--- a/test/org/apache/tomcat/util/http/TestFastHttpDateFormatPerformance.java
+++ b/test/org/apache/tomcat/util/http/TesterFastHttpDateFormatPerformance.java
@@ -18,7 +18,11 @@ package org.apache.tomcat.util.http;
import org.junit.Test;
-public class TestFastHttpDateFormatPerformance {
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
+public class TesterFastHttpDateFormatPerformance {
@Test
public void testGetCurrentDateConcurrent() throws InterruptedException {
diff --git a/test/org/apache/tomcat/websocket/TestConnectionLimitPerformance.java b/test/org/apache/tomcat/websocket/TesterConnectionLimitPerformance.java
similarity index 93%
rename from test/org/apache/tomcat/websocket/TestConnectionLimitPerformance.java
rename to test/org/apache/tomcat/websocket/TesterConnectionLimitPerformance.java
index df4b43e9ce..4d2a801e51 100644
--- a/test/org/apache/tomcat/websocket/TestConnectionLimitPerformance.java
+++ b/test/org/apache/tomcat/websocket/TesterConnectionLimitPerformance.java
@@ -33,7 +33,11 @@ import org.apache.catalina.startup.Tomcat;
import org.apache.catalina.startup.TomcatBaseTest;
import org.apache.tomcat.websocket.TesterMessageCountClient.TesterProgrammaticEndpoint;
-public class TestConnectionLimitPerformance extends TomcatBaseTest {
+/*
+ * This test runs until the test machine runs out of resources. There is no benefit it running it as part of a standard
+ * test run so it is excluded due to the name starting Tester...
+ */
+public class TesterConnectionLimitPerformance extends TomcatBaseTest {
/*
* Simple test to see how many outgoing connections can be created on a single machine.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
(tomcat) 04/08: Add additional comment to this relative performance test
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 4332382bb11d86197bfce2e9ca14c75a3c433225
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Mon Jan 8 12:08:41 2024 +0000
Add additional comment to this relative performance test
---
test/org/apache/catalina/connector/TestResponsePerformance.java | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/test/org/apache/catalina/connector/TestResponsePerformance.java b/test/org/apache/catalina/connector/TestResponsePerformance.java
index eb1236e80b..e88164c59a 100644
--- a/test/org/apache/catalina/connector/TestResponsePerformance.java
+++ b/test/org/apache/catalina/connector/TestResponsePerformance.java
@@ -24,6 +24,10 @@ import org.junit.Test;
import org.apache.catalina.startup.LoggingBaseTest;
import org.apache.tomcat.unittest.TesterRequest;
+/*
+ * This is a relative performance test so it remains part of the standard test run. If the test fails then we need to
+ * looking at why and possibly disable/remove the homebrew approach for some OS/Java combinations.
+ */
public class TestResponsePerformance extends LoggingBaseTest {
private static final int ITERATIONS = 100000;
@@ -38,9 +42,8 @@ public class TestResponsePerformance extends LoggingBaseTest {
doHomebrew(resp);
doUri();
- // Note: Java 9 on my OSX laptop consistently shows doUri() is faster
- // than doHomebrew(). Worth a closer look for Tomcat 10 on the
- // assumption it will require java 9
+ // Note: With Java 11 the 'homebrew' approach is consistently 3-4 times faster on both MacOS (Intel) and Linux
+ // With Java 22 EA the 'homebrew' approach is consistently a little over 2x faster on MacOS (M1)
// To allow for timing differences between runs, a "best of n" approach
// is taken for this test
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
(tomcat) 07/08: Update the docs for running the tests
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit f5213adc3069f09c609ae602efa01adb7eca6b60
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Mon Jan 8 12:11:41 2024 +0000
Update the docs for running the tests
---
BUILDING.txt | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/BUILDING.txt b/BUILDING.txt
index 5e16755e96..4fd85f1266 100644
--- a/BUILDING.txt
+++ b/BUILDING.txt
@@ -491,9 +491,20 @@ NOTE: Cobertura is licensed under GPL v2 with parts of it being under
(such as a developer may use day to day) assuming no other resource hungry
processes are running.
- These assumptions are not always true (e.g. on CI systems running in a
- virtual machine) so the performance tests may be deactivated by using the
- following property:
+ Performance tests may be an absolute test (how long to complete a number
+ of iterations of an operation or set of operations) or may be a relative
+ test (how long two or more different approaches take to generate the same
+ result). The absolute tests may be destructive in that they run until the
+ system runs out of resources.
+
+ Where there is no benefit in running an absolute performance test as part
+ of a standard test run, the test will be excluded by naming it
+ Tester*Performance.java.
+
+ The relative tests are included as part of a standard test run however,
+ where the assumptions made about host capabilities are not true (e.g. on
+ CI systems running in virtual machine) the tests may be deactivated by
+ using the following property:
test.excludePerformance=true
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
(tomcat) 03/08: Refactor so these relative performance tests generate pass/fail results
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit aebc402308129424590f32fa8b9f3285451d3033
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Mon Jan 8 12:08:21 2024 +0000
Refactor so these relative performance tests generate pass/fail results
---
.../apache/el/parser/TestELParserPerformance.java | 10 +++++++
.../juli/TestOneLineFormatterPerformance.java | 31 +++++++++++++++-------
.../util/buf/TestCharsetCachePerformance.java | 30 ++++++++++-----------
3 files changed, 46 insertions(+), 25 deletions(-)
diff --git a/test/org/apache/el/parser/TestELParserPerformance.java b/test/org/apache/el/parser/TestELParserPerformance.java
index f4d90e7f9e..6a282cded6 100644
--- a/test/org/apache/el/parser/TestELParserPerformance.java
+++ b/test/org/apache/el/parser/TestELParserPerformance.java
@@ -18,10 +18,14 @@ package org.apache.el.parser;
import java.io.StringReader;
+import org.junit.Assert;
import org.junit.Test;
import org.apache.tomcat.util.collections.SynchronizedStack;
+/*
+ * This is a relative performance test so it remains part of the standard test run.
+ */
public class TestELParserPerformance {
/*
@@ -46,6 +50,8 @@ public class TestELParserPerformance {
final int runs = 20;
final int parseIterations = 10000;
+ long reinitTotalTime = 0;
+ long newTotalTime = 0;
for (int j = 0; j < runs; j ++) {
long start = System.nanoTime();
@@ -62,6 +68,7 @@ public class TestELParserPerformance {
stack.push(parser);
}
long end = System.nanoTime();
+ reinitTotalTime += (end - start);
System.out.println(parseIterations +
" iterations using ELParser.ReInit(...) took " + (end - start) + "ns");
@@ -74,9 +81,12 @@ public class TestELParserPerformance {
parser.CompositeExpression();
}
long end = System.nanoTime();
+ newTotalTime += (end - start);
System.out.println(parseIterations +
" iterations using new ELParser(...) took " + (end - start) + "ns");
}
+
+ Assert.assertTrue("Using new ElParser() was faster then using ELParser.ReInit", reinitTotalTime < newTotalTime);
}
}
diff --git a/test/org/apache/juli/TestOneLineFormatterPerformance.java b/test/org/apache/juli/TestOneLineFormatterPerformance.java
index 54b1e33ebd..a90ebd6529 100644
--- a/test/org/apache/juli/TestOneLineFormatterPerformance.java
+++ b/test/org/apache/juli/TestOneLineFormatterPerformance.java
@@ -16,6 +16,7 @@
*/
package org.apache.juli;
+import org.junit.Assert;
import org.junit.Test;
/**
@@ -28,19 +29,29 @@ public class TestOneLineFormatterPerformance {
@Test
public void testDateFormat() throws Exception {
+ DateFormat stringFormatImpl = new StringFormatImpl();
+ long stringFormatImplTime = doTestDateFormat(stringFormatImpl);
+
+ DateFormat dateFormatCacheImpl = new DateFormatCacheImpl();
+ long dateFormatCacheImplTime = doTestDateFormat(dateFormatCacheImpl);
+
+ Assert.assertTrue("String#format was faster that DateFormatCache",
+ dateFormatCacheImplTime < stringFormatImplTime);
+ }
+
+
+ private long doTestDateFormat(DateFormat df) {
int iters = 1000000;
- DateFormat[] dfs = new DateFormat[] { new StringFormatImpl(), new DateFormatCacheImpl() };
-
- for (DateFormat df : dfs) {
- long start = System.nanoTime();
- for (int i = 0; i < iters; i++) {
- df.format(System.nanoTime());
- }
- long end = System.nanoTime();
- System.out.println(
- "Impl: [" + df.getClass().getName() + "] took [" + (end-start) + "] ns");
+
+ long start = System.nanoTime();
+ for (int i = 0; i < iters; i++) {
+ df.format(System.nanoTime());
}
+ long end = System.nanoTime();
+ System.out.println(
+ "Impl: [" + df.getClass().getName() + "] took [" + (end - start) + "] ns");
+ return end - start;
}
diff --git a/test/org/apache/tomcat/util/buf/TestCharsetCachePerformance.java b/test/org/apache/tomcat/util/buf/TestCharsetCachePerformance.java
index 2ad8f7f142..f17098488e 100644
--- a/test/org/apache/tomcat/util/buf/TestCharsetCachePerformance.java
+++ b/test/org/apache/tomcat/util/buf/TestCharsetCachePerformance.java
@@ -21,29 +21,27 @@ import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
+import org.junit.Assert;
import org.junit.Test;
+/*
+ * This is a relative performance test so it remains part of the standard test run.
+ */
public class TestCharsetCachePerformance {
@Test
- public void testNoCsCache() throws Exception {
- doTest(new NoCsCache());
- }
-
-
- @Test
- public void testFullCsCache() throws Exception {
- doTest(new FullCsCache());
+ public void testCache() throws Exception {
+ long timeNone = doTest(new NoCsCache());
+ long timeFull = doTest(new FullCsCache());
+ long timeLazy = doTest(new LazyCsCache());
+
+ Assert.assertTrue("No cache was faster than full cache", timeFull < timeNone);
+ Assert.assertTrue("No cache was faster than lazy cache", timeLazy < timeNone);
+ Assert.assertTrue("Lazy cache was faster than full cache ", timeFull < timeLazy);
}
- @Test
- public void testLazyCsCache() throws Exception {
- doTest(new LazyCsCache());
- }
-
-
- private void doTest(CsCache cache) throws Exception {
+ private long doTest(CsCache cache) throws Exception {
int threadCount = 10;
int iterations = 10000000;
String[] lookupNames = new String[] {
@@ -68,6 +66,8 @@ public class TestCharsetCachePerformance {
long endTime = System.nanoTime();
System.out.println(cache.getClass().getName() + ": " + (endTime - startTime) + "ns");
+
+ return endTime - startTime;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
(tomcat) 05/08: Comment this absolute performance test with an upper limit
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 5d04b808995aa16539edf6f89c72b4ecef9b3f4d
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Mon Jan 8 12:09:29 2024 +0000
Comment this absolute performance test with an upper limit
Since there is an upper limit this test can pass/fail so it is included
in the standard test run.
---
test/org/apache/catalina/mapper/TestMapperPerformance.java | 3 +++
1 file changed, 3 insertions(+)
diff --git a/test/org/apache/catalina/mapper/TestMapperPerformance.java b/test/org/apache/catalina/mapper/TestMapperPerformance.java
index 606794397f..4b4feb8f81 100644
--- a/test/org/apache/catalina/mapper/TestMapperPerformance.java
+++ b/test/org/apache/catalina/mapper/TestMapperPerformance.java
@@ -21,6 +21,9 @@ import org.junit.Test;
import org.apache.tomcat.util.buf.MessageBytes;
+/*
+ * This is an absolute performance test with an upper limit. Therefore it is executed as part of a standard test run.
+ */
public class TestMapperPerformance extends TestMapper {
@Test
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
(tomcat) 02/08: Add additional comment to existing absolute performance tests
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 4627204ed27ba99e65bcab0c878d7dc940aea38c
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Mon Jan 8 12:06:53 2024 +0000
Add additional comment to existing absolute performance tests
---
test/javax/el/TesterImportHandlerPerformance.java | 4 ++++
test/javax/websocket/TesterContainerProviderPerformance.java | 4 ++++
.../authenticator/TesterDigestAuthenticatorPerformance.java | 4 ++++
test/org/apache/tomcat/unittest/TesterThreadedPerformance.java | 3 +++
test/org/apache/tomcat/util/bcel/TesterPerformance.java | 6 ++++++
test/org/apache/tomcat/util/http/TesterCookiesPerformance.java | 4 ++++
test/org/apache/tomcat/util/http/TesterParametersPerformance.java | 4 ++++
7 files changed, 29 insertions(+)
diff --git a/test/javax/el/TesterImportHandlerPerformance.java b/test/javax/el/TesterImportHandlerPerformance.java
index ce05d0b8d0..77df4f2bd8 100644
--- a/test/javax/el/TesterImportHandlerPerformance.java
+++ b/test/javax/el/TesterImportHandlerPerformance.java
@@ -18,6 +18,10 @@ package javax.el;
import org.junit.Test;
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
public class TesterImportHandlerPerformance {
/*
diff --git a/test/javax/websocket/TesterContainerProviderPerformance.java b/test/javax/websocket/TesterContainerProviderPerformance.java
index 92b617a0a8..4732a834f8 100644
--- a/test/javax/websocket/TesterContainerProviderPerformance.java
+++ b/test/javax/websocket/TesterContainerProviderPerformance.java
@@ -23,6 +23,10 @@ import org.junit.Test;
import org.apache.tomcat.unittest.TesterThreadedPerformance;
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
public class TesterContainerProviderPerformance {
@Test
diff --git a/test/org/apache/catalina/authenticator/TesterDigestAuthenticatorPerformance.java b/test/org/apache/catalina/authenticator/TesterDigestAuthenticatorPerformance.java
index c5006b5317..b99b8b1f63 100644
--- a/test/org/apache/catalina/authenticator/TesterDigestAuthenticatorPerformance.java
+++ b/test/org/apache/catalina/authenticator/TesterDigestAuthenticatorPerformance.java
@@ -41,6 +41,10 @@ import org.apache.tomcat.util.buf.HexUtils;
import org.apache.tomcat.util.descriptor.web.LoginConfig;
import org.apache.tomcat.util.security.ConcurrentMessageDigest;
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
public class TesterDigestAuthenticatorPerformance {
private static String USER = "user";
diff --git a/test/org/apache/tomcat/unittest/TesterThreadedPerformance.java b/test/org/apache/tomcat/unittest/TesterThreadedPerformance.java
index 2c3112ca69..9e0ea4c29e 100644
--- a/test/org/apache/tomcat/unittest/TesterThreadedPerformance.java
+++ b/test/org/apache/tomcat/unittest/TesterThreadedPerformance.java
@@ -19,6 +19,9 @@ package org.apache.tomcat.unittest;
import java.util.function.IntConsumer;
import java.util.function.Supplier;
+/*
+ * Support class for threaded performance tests.
+ */
public class TesterThreadedPerformance {
private final int threadCount;
diff --git a/test/org/apache/tomcat/util/bcel/TesterPerformance.java b/test/org/apache/tomcat/util/bcel/TesterPerformance.java
index 9319c47e2e..de785576a5 100644
--- a/test/org/apache/tomcat/util/bcel/TesterPerformance.java
+++ b/test/org/apache/tomcat/util/bcel/TesterPerformance.java
@@ -31,6 +31,12 @@ import org.apache.tomcat.Jar;
import org.apache.tomcat.util.bcel.classfile.ClassParser;
import org.apache.tomcat.util.scan.JarFactory;
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ *
+ * The test also requires that the Jira lib directory has been extracted from a Jira install and copied to /tmp.
+ */
public class TesterPerformance {
private static final String JAR_LOCATION = "/tmp/jira-libs";
diff --git a/test/org/apache/tomcat/util/http/TesterCookiesPerformance.java b/test/org/apache/tomcat/util/http/TesterCookiesPerformance.java
index 4f186c705d..3e3f2fb569 100644
--- a/test/org/apache/tomcat/util/http/TesterCookiesPerformance.java
+++ b/test/org/apache/tomcat/util/http/TesterCookiesPerformance.java
@@ -21,6 +21,10 @@ import org.junit.Test;
import org.apache.tomcat.util.buf.MessageBytes;
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
public class TesterCookiesPerformance {
@Test
diff --git a/test/org/apache/tomcat/util/http/TesterParametersPerformance.java b/test/org/apache/tomcat/util/http/TesterParametersPerformance.java
index 17a7d5f8ec..abcc47f9e9 100644
--- a/test/org/apache/tomcat/util/http/TesterParametersPerformance.java
+++ b/test/org/apache/tomcat/util/http/TesterParametersPerformance.java
@@ -27,6 +27,10 @@ import org.junit.Test;
import org.apache.tomcat.util.buf.B2CConverter;
+/*
+ * This is an absolute performance test. There is no benefit it running it as part of a standard test run so it is
+ * excluded due to the name starting Tester...
+ */
public class TesterParametersPerformance {
@Test
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org