You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by ma...@apache.org on 2023/05/09 20:02:10 UTC
[tomcat] branch 8.5.x updated (eb85bb201a -> 31a7711d1b)
This is an automated email from the ASF dual-hosted git repository.
markt pushed a change to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
from eb85bb201a Make checkstyle actually happy.
new bb0df62ab3 Fix IDE warnings
new b4f09a0082 Clean-up - formatting. No functional change.
new 31a7711d1b MacOS seems to need longer for the test to complete
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/catalina/filters/RateLimitFilter.java | 90 ++++++++++------------
.../apache/catalina/util/TimeBucketCounter.java | 49 +++++-------
.../catalina/filters/TestRateLimitFilter.java | 32 ++++----
3 files changed, 80 insertions(+), 91 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[tomcat] 02/03: Clean-up - formatting. No functional change.
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit b4f09a0082acac7da9842d4cb003ff752917915e
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Tue May 9 20:38:49 2023 +0100
Clean-up - formatting. No functional change.
---
.../apache/catalina/filters/RateLimitFilter.java | 90 ++++++++++------------
.../apache/catalina/util/TimeBucketCounter.java | 48 +++++-------
.../catalina/filters/TestRateLimitFilter.java | 14 ++--
3 files changed, 66 insertions(+), 86 deletions(-)
diff --git a/java/org/apache/catalina/filters/RateLimitFilter.java b/java/org/apache/catalina/filters/RateLimitFilter.java
index f3ded50cf0..2b35243e84 100644
--- a/java/org/apache/catalina/filters/RateLimitFilter.java
+++ b/java/org/apache/catalina/filters/RateLimitFilter.java
@@ -33,45 +33,41 @@ import org.apache.juli.logging.LogFactory;
import org.apache.tomcat.util.res.StringManager;
/**
- * <p>Servlet filter that can help mitigate Denial of Service
- * (DoS) and Brute Force attacks by limiting the number of a requests that are
- * allowed from a single IP address within a time window (also referred
- * to as a time bucket), e.g. 300 Requests per 60 seconds.</p>
- *
- * <p>The filter works by incrementing a counter in a time bucket for each IP
- * address, and if the counter exceeds the allowed limit then further requests
- * from that IP are dropped with a "429 Too many requests" response
- * until the bucket time ends and a new bucket starts.</p>
- *
- * <p>The filter is optimized for efficiency and low overhead, so it converts
- * some configured values to more efficient values. For example, a configuration
- * of a 60 seconds time bucket is converted to 65.536 seconds. That allows
- * for very fast bucket calculation using bit shift arithmetic. In order to remain
- * true to the user intent, the configured number of requests is then multiplied
- * by the same ratio, so a configuration of 100 Requests per 60 seconds, has the
- * real values of 109 Requests per 65 seconds.</p>
- *
- * <p>It is common to set up different restrictions for different URIs.
- * For example, a login page or authentication script is typically expected
- * to get far less requests than the rest of the application, so you can add
- * a filter definition that would allow only 5 requests per 15 seconds and map
- * those URIs to it.</p>
- *
- * <p>You can set <code>enforce</code> to <code>false</code>
- * to disable the termination of requests that exceed the allowed limit. Then
- * your application code can inspect the Request Attribute
- * <code>org.apache.catalina.filters.RateLimitFilter.Count</code> and decide
- * how to handle the request based on other information that it has, e.g. allow
- * more requests to certain users based on roles, etc.</p>
- *
- * <p><strong>WARNING:</strong> if Tomcat is behind a reverse proxy then you must
- * make sure that the Rate Limit Filter sees the client IP address, so if for
- * example you are using the <a href="#Remote_IP_Filter">Remote IP Filter</a>,
- * then the filter mapping for the Rate Limit Filter must come <em>after</em>
- * the mapping of the Remote IP Filter to ensure that each request has its IP
- * address resolved before the Rate Limit Filter is applied. Failure to do so
- * will count requests from different IPs in the same bucket and will result in
- * a self inflicted DoS attack.</p>
+ * <p>
+ * Servlet filter that can help mitigate Denial of Service (DoS) and Brute Force attacks by limiting the number of a
+ * requests that are allowed from a single IP address within a time window (also referred to as a time bucket), e.g. 300
+ * Requests per 60 seconds.
+ * </p>
+ * <p>
+ * The filter works by incrementing a counter in a time bucket for each IP address, and if the counter exceeds the
+ * allowed limit then further requests from that IP are dropped with a "429 Too many requests" response until
+ * the bucket time ends and a new bucket starts.
+ * </p>
+ * <p>
+ * The filter is optimized for efficiency and low overhead, so it converts some configured values to more efficient
+ * values. For example, a configuration of a 60 seconds time bucket is converted to 65.536 seconds. That allows for very
+ * fast bucket calculation using bit shift arithmetic. In order to remain true to the user intent, the configured number
+ * of requests is then multiplied by the same ratio, so a configuration of 100 Requests per 60 seconds, has the real
+ * values of 109 Requests per 65 seconds.
+ * </p>
+ * <p>
+ * It is common to set up different restrictions for different URIs. For example, a login page or authentication script
+ * is typically expected to get far less requests than the rest of the application, so you can add a filter definition
+ * that would allow only 5 requests per 15 seconds and map those URIs to it.
+ * </p>
+ * <p>
+ * You can set <code>enforce</code> to <code>false</code> to disable the termination of requests that exceed the allowed
+ * limit. Then your application code can inspect the Request Attribute
+ * <code>org.apache.catalina.filters.RateLimitFilter.Count</code> and decide how to handle the request based on other
+ * information that it has, e.g. allow more requests to certain users based on roles, etc.
+ * </p>
+ * <p>
+ * <strong>WARNING:</strong> if Tomcat is behind a reverse proxy then you must make sure that the Rate Limit Filter sees
+ * the client IP address, so if for example you are using the <a href="#Remote_IP_Filter">Remote IP Filter</a>, then the
+ * filter mapping for the Rate Limit Filter must come <em>after</em> the mapping of the Remote IP Filter to ensure that
+ * each request has its IP address resolved before the Rate Limit Filter is applied. Failure to do so will count
+ * requests from different IPs in the same bucket and will result in a self inflicted DoS attack.
+ * </p>
*/
public class RateLimitFilter implements Filter {
@@ -199,16 +195,14 @@ public class RateLimitFilter implements Filter {
actualRequests = (int) Math.round(bucketCounter.getRatio() * bucketRequests);
- log.info(sm.getString("rateLimitFilter.initialized",
- filterName, Integer.valueOf(bucketRequests), Integer.valueOf(bucketDuration),
- Integer.valueOf(getActualRequests()), Integer.valueOf(getActualDurationInSeconds()),
- (!enforce ? "Not " : "") + "enforcing")
- );
+ log.info(sm.getString("rateLimitFilter.initialized", filterName, Integer.valueOf(bucketRequests),
+ Integer.valueOf(bucketDuration), Integer.valueOf(getActualRequests()),
+ Integer.valueOf(getActualDurationInSeconds()), (!enforce ? "Not " : "") + "enforcing"));
}
@Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
- throws IOException, ServletException {
+ throws IOException, ServletException {
String ipAddr = request.getRemoteAddr();
int reqCount = bucketCounter.increment(ipAddr);
@@ -218,10 +212,8 @@ public class RateLimitFilter implements Filter {
if (enforce && (reqCount > actualRequests)) {
((HttpServletResponse) response).sendError(statusCode, statusMessage);
- log.warn(sm.getString("rateLimitFilter.maxRequestsExceeded",
- filterName, Integer.valueOf(reqCount), ipAddr, Integer.valueOf(getActualRequests()),
- Integer.valueOf(getActualDurationInSeconds()))
- );
+ log.warn(sm.getString("rateLimitFilter.maxRequestsExceeded", filterName, Integer.valueOf(reqCount), ipAddr,
+ Integer.valueOf(getActualRequests()), Integer.valueOf(getActualDurationInSeconds())));
return;
}
diff --git a/java/org/apache/catalina/util/TimeBucketCounter.java b/java/org/apache/catalina/util/TimeBucketCounter.java
index d104eebf4e..ef93f51665 100644
--- a/java/org/apache/catalina/util/TimeBucketCounter.java
+++ b/java/org/apache/catalina/util/TimeBucketCounter.java
@@ -22,24 +22,19 @@ import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
/**
- * this class maintains a thread safe hash map that has timestamp-based buckets
- * followed by a string for a key, and a counter for a value. each time the
- * increment() method is called it adds the key if it does not exist, increments
- * its value and returns it.
- *
- * a maintenance thread cleans up keys that are prefixed by previous timestamp
- * buckets.
+ * this class maintains a thread safe hash map that has timestamp-based buckets followed by a string for a key, and a
+ * counter for a value. each time the increment() method is called it adds the key if it does not exist, increments its
+ * value and returns it. a maintenance thread cleans up keys that are prefixed by previous timestamp buckets.
*/
public class TimeBucketCounter {
/**
* Map to hold the buckets
*/
- private final ConcurrentHashMap<String, AtomicInteger> map = new ConcurrentHashMap<>();
+ private final ConcurrentHashMap<String,AtomicInteger> map = new ConcurrentHashMap<>();
/**
- * Milliseconds bucket size as a Power of 2 for bit shift math, e.g.
- * 16 for 65_536ms which is about 1:05 minute
+ * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. 16 for 65_536ms which is about 1:05 minute
*/
private final int numBits;
@@ -79,10 +74,10 @@ public class TimeBucketCounter {
}
/**
- * increments the counter for the passed identifier in the current time
- * bucket and returns the new value
+ * increments the counter for the passed identifier in the current time bucket and returns the new value
*
* @param identifier an identifier for which we want to maintain count, e.g. IP Address
+ *
* @return the count within the current time bucket
*/
public final int increment(String identifier) {
@@ -100,9 +95,8 @@ public class TimeBucketCounter {
}
/**
- * calculates the current time bucket prefix by shifting bits for fast
- * division, e.g. shift 16 bits is the same as dividing by 65,536 which is
- * about 1:05m
+ * calculates the current time bucket prefix by shifting bits for fast division, e.g. shift 16 bits is the same as
+ * dividing by 65,536 which is about 1:05m
*
* @return The current bucket prefix.
*/
@@ -115,9 +109,8 @@ public class TimeBucketCounter {
}
/**
- * the actual duration may differ from the configured duration because
- * it is set to the next power of 2 value in order to perform very fast
- * bit shift arithmetic
+ * the actual duration may differ from the configured duration because it is set to the next power of 2 value in
+ * order to perform very fast bit shift arithmetic
*
* @return the actual bucket duration in milliseconds
*/
@@ -126,13 +119,11 @@ public class TimeBucketCounter {
}
/**
- * returns the ratio between the configured duration param and the
- * actual duration which will be set to the next power of 2. we then
- * multiply the configured requests param by the same ratio in order
- * to compensate for the added time, if any
+ * returns the ratio between the configured duration param and the actual duration which will be set to the next
+ * power of 2. we then multiply the configured requests param by the same ratio in order to compensate for the added
+ * time, if any
*
- * @return the ratio, e.g. 1.092 if the actual duration is 65_536 for
- * the configured duration of 60_000
+ * @return the ratio, e.g. 1.092 if the actual duration is 65_536 for the configured duration of 60_000
*/
public double getRatio() {
return ratio;
@@ -147,8 +138,7 @@ public class TimeBucketCounter {
}
/**
- * returns the next power of 2 given a value, e.g. 256 for 250,
- * or 1024, for 1000
+ * returns the next power of 2 given a value, e.g. 256 for 250, or 1024, for 1000
*/
static int nextPowerOf2(int value) {
int valueOfHighestBit = Integer.highestOneBit(value);
@@ -160,8 +150,7 @@ public class TimeBucketCounter {
}
/**
- * when we want to test a full bucket duration we need to sleep until the
- * next bucket starts
+ * when we want to test a full bucket duration we need to sleep until the next bucket starts
*
* @return the number of milliseconds until the next bucket
*/
@@ -214,7 +203,8 @@ public class TimeBucketCounter {
try {
Thread.sleep(sleeptime);
- } catch (InterruptedException e) {}
+ } catch (InterruptedException e) {
+ }
}
}
}
diff --git a/test/org/apache/catalina/filters/TestRateLimitFilter.java b/test/org/apache/catalina/filters/TestRateLimitFilter.java
index 684bd3b258..aa6ffd0c48 100644
--- a/test/org/apache/catalina/filters/TestRateLimitFilter.java
+++ b/test/org/apache/catalina/filters/TestRateLimitFilter.java
@@ -73,15 +73,15 @@ public class TestRateLimitFilter extends TomcatBaseTest {
Thread.sleep(5000);
- Assert.assertEquals(200, tc1.results[24]); // only 25 requests made, all allowed
+ Assert.assertEquals(200, tc1.results[24]); // only 25 requests made, all allowed
- Assert.assertEquals(200, tc2.results[49]); // only 25 requests made, all allowed
+ Assert.assertEquals(200, tc2.results[49]); // only 25 requests made, all allowed
Assert.assertEquals(200, tc3.results[allowedRequests - 1]); // first allowedRequests allowed
- Assert.assertEquals(429, tc3.results[allowedRequests]); // subsequent requests dropped
+ Assert.assertEquals(429, tc3.results[allowedRequests]); // subsequent requests dropped
Assert.assertEquals(200, tc4.results[allowedRequests - 1]); // first allowedRequests allowed
- Assert.assertEquals(429, tc4.results[allowedRequests]); // subsequent requests dropped
+ Assert.assertEquals(429, tc4.results[allowedRequests]); // subsequent requests dropped
}
private RateLimitFilter testRateLimitFilter(FilterDef filterDef, Context root) throws ServletException {
@@ -102,7 +102,6 @@ public class TestRateLimitFilter extends TomcatBaseTest {
rateLimitFilter.init(filterConfig);
return rateLimitFilter;
- //*/
}
static class TestClient extends Thread {
@@ -140,8 +139,7 @@ public class TestRateLimitFilter extends TomcatBaseTest {
Integer.valueOf(response.getStatus()));
Thread.sleep(sleep);
}
- }
- catch (Exception ex) {
+ } catch (Exception ex) {
ex.printStackTrace();
}
}
@@ -167,7 +165,7 @@ public class TestRateLimitFilter extends TomcatBaseTest {
private static FilterConfig generateFilterConfig(FilterDef filterDef) {
final TesterServletContext mockServletContext = new TesterServletContext();
- final Map<String, String> parameters = filterDef.getParameterMap();
+ final Map<String,String> parameters = filterDef.getParameterMap();
FilterConfig filterConfig = new FilterConfig() {
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[tomcat] 01/03: Fix IDE warnings
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit bb0df62ab311d8f264d88aa7c2ec91285a68d7ac
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Tue May 9 20:32:29 2023 +0100
Fix IDE warnings
---
java/org/apache/catalina/filters/RateLimitFilter.java | 10 ++++++----
java/org/apache/catalina/util/TimeBucketCounter.java | 1 +
test/org/apache/catalina/filters/TestRateLimitFilter.java | 9 ++++-----
3 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/java/org/apache/catalina/filters/RateLimitFilter.java b/java/org/apache/catalina/filters/RateLimitFilter.java
index 32262b8b2a..f3ded50cf0 100644
--- a/java/org/apache/catalina/filters/RateLimitFilter.java
+++ b/java/org/apache/catalina/filters/RateLimitFilter.java
@@ -200,8 +200,9 @@ public class RateLimitFilter implements Filter {
actualRequests = (int) Math.round(bucketCounter.getRatio() * bucketRequests);
log.info(sm.getString("rateLimitFilter.initialized",
- filterName, bucketRequests, bucketDuration, getActualRequests(),
- getActualDurationInSeconds(), (!enforce ? "Not " : "") + "enforcing")
+ filterName, Integer.valueOf(bucketRequests), Integer.valueOf(bucketDuration),
+ Integer.valueOf(getActualRequests()), Integer.valueOf(getActualDurationInSeconds()),
+ (!enforce ? "Not " : "") + "enforcing")
);
}
@@ -212,13 +213,14 @@ public class RateLimitFilter implements Filter {
String ipAddr = request.getRemoteAddr();
int reqCount = bucketCounter.increment(ipAddr);
- request.setAttribute(RATE_LIMIT_ATTRIBUTE_COUNT, reqCount);
+ request.setAttribute(RATE_LIMIT_ATTRIBUTE_COUNT, Integer.valueOf(reqCount));
if (enforce && (reqCount > actualRequests)) {
((HttpServletResponse) response).sendError(statusCode, statusMessage);
log.warn(sm.getString("rateLimitFilter.maxRequestsExceeded",
- filterName, reqCount, ipAddr, getActualRequests(), getActualDurationInSeconds())
+ filterName, Integer.valueOf(reqCount), ipAddr, Integer.valueOf(getActualRequests()),
+ Integer.valueOf(getActualDurationInSeconds()))
);
return;
diff --git a/java/org/apache/catalina/util/TimeBucketCounter.java b/java/org/apache/catalina/util/TimeBucketCounter.java
index 5c602c96af..d104eebf4e 100644
--- a/java/org/apache/catalina/util/TimeBucketCounter.java
+++ b/java/org/apache/catalina/util/TimeBucketCounter.java
@@ -191,6 +191,7 @@ public class TimeBucketCounter {
this.sleeptime = sleeptime;
}
+ @SuppressWarnings("sync-override")
@Override
public void start() {
isRunning = true;
diff --git a/test/org/apache/catalina/filters/TestRateLimitFilter.java b/test/org/apache/catalina/filters/TestRateLimitFilter.java
index 9f82306367..684bd3b258 100644
--- a/test/org/apache/catalina/filters/TestRateLimitFilter.java
+++ b/test/org/apache/catalina/filters/TestRateLimitFilter.java
@@ -31,7 +31,6 @@ import org.junit.Assert;
import org.junit.Test;
import org.apache.catalina.Context;
-import org.apache.catalina.LifecycleException;
import org.apache.catalina.filters.TestRemoteIpFilter.MockFilterChain;
import org.apache.catalina.filters.TestRemoteIpFilter.MockHttpServletRequest;
import org.apache.catalina.startup.Tomcat;
@@ -63,7 +62,7 @@ public class TestRateLimitFilter extends TomcatBaseTest {
int allowedRequests = (int) Math.round(rateLimitFilter.bucketCounter.getRatio() * bucketRequests);
long sleepTime = rateLimitFilter.bucketCounter.getMillisUntilNextBucket();
- System.out.printf("Sleeping %d millis for the next time bucket to start\n", sleepTime);
+ System.out.printf("Sleeping %d millis for the next time bucket to start\n", Long.valueOf(sleepTime));
Thread.sleep(sleepTime);
TestClient tc1 = new TestClient(rateLimitFilter, filterChain, "10.20.20.5", 200, 5);
@@ -85,8 +84,7 @@ public class TestRateLimitFilter extends TomcatBaseTest {
Assert.assertEquals(429, tc4.results[allowedRequests]); // subsequent requests dropped
}
- private RateLimitFilter testRateLimitFilter(FilterDef filterDef, Context root)
- throws LifecycleException, IOException, ServletException {
+ private RateLimitFilter testRateLimitFilter(FilterDef filterDef, Context root) throws ServletException {
RateLimitFilter rateLimitFilter = new RateLimitFilter();
filterDef.setFilterClass(RateLimitFilter.class.getName());
@@ -138,7 +136,8 @@ public class TestRateLimitFilter extends TomcatBaseTest {
response.setRequest(request);
filter.doFilter(request, response, filterChain);
results[i] = response.getStatus();
- System.out.printf("%s %s: %s %d\n", ip, new Date(), i + 1, response.getStatus());
+ System.out.printf("%s %s: %s %d\n", ip, new Date(), Integer.valueOf(i + 1),
+ Integer.valueOf(response.getStatus()));
Thread.sleep(sleep);
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[tomcat] 03/03: MacOS seems to need longer for the test to complete
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 31a7711d1bd4ae4e00c9267c478e695ad07a820b
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Tue May 9 20:41:53 2023 +0100
MacOS seems to need longer for the test to complete
---
test/org/apache/catalina/filters/TestRateLimitFilter.java | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/test/org/apache/catalina/filters/TestRateLimitFilter.java b/test/org/apache/catalina/filters/TestRateLimitFilter.java
index aa6ffd0c48..7e986e75d0 100644
--- a/test/org/apache/catalina/filters/TestRateLimitFilter.java
+++ b/test/org/apache/catalina/filters/TestRateLimitFilter.java
@@ -71,7 +71,14 @@ public class TestRateLimitFilter extends TomcatBaseTest {
TestClient tc3 = new TestClient(rateLimitFilter, filterChain, "10.20.20.20", 200, 20);
TestClient tc4 = new TestClient(rateLimitFilter, filterChain, "10.20.20.40", 200, 40);
- Thread.sleep(5000);
+ // Sleep for up to 10s for clients to complete
+ int count = 0;
+ while (count < 100 && (tc1.results[24] == 0 || tc2.results[49] == 0 || tc3.results[allowedRequests - 1] == 0 ||
+ tc3.results[allowedRequests] == 0 || tc3.results[allowedRequests - 1] == 0 ||
+ tc4.results[allowedRequests] == 0)) {
+ Thread.sleep(100);
+ count++;
+ }
Assert.assertEquals(200, tc1.results[24]); // only 25 requests made, all allowed
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org