You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by bt...@apache.org on 2022/01/27 02:08:46 UTC
[james-project] branch master updated (21c7af7 -> e019bea)
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git.
from 21c7af7 JAMES-3693 Increase precision in tests
new 29fd7c6 JAMES-3693 Allow instanciation of MemoryRateLimiter from within mailets
new 22cc871 JAMES-3683 Implement PerSenderRateLimit mailet
new b8eb299 JAMES-3683 Docker-compose demonstrating the setup of PerSenderRateLimit mailet
new 2e2a694 JAMES-3683 Reduce size of the rate limiting plugin by excluding James dependencies
new e019bea JAMES-3683 RateLimiter: Allow using longs for allowed quantities
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
server/mailet/rate-limiter/README.adoc | 61 ++
server/mailet/rate-limiter/docker-compose.yml | 12 +
server/mailet/rate-limiter/extensions.properties | 1 +
.../rate-limiter}/mailetcontainer.xml | 8 +
server/mailet/rate-limiter/pom.xml | 34 ++
.../james/rate/limiter/api/RateLimiter.scala | 49 +-
.../rate/limiter/memory/MemoryRateLimiter.scala | 18 +-
.../transport/mailets/PerSenderRateLimit.scala | 220 ++++++++
.../rate/limiter/api/RateLimiterContract.scala | 16 +-
.../transport/mailets/PerSenderRateLimitTest.scala | 624 +++++++++++++++++++++
10 files changed, 1022 insertions(+), 21 deletions(-)
create mode 100644 server/mailet/rate-limiter/README.adoc
create mode 100644 server/mailet/rate-limiter/docker-compose.yml
create mode 100644 server/mailet/rate-limiter/extensions.properties
copy server/{apps/memory-app/sample-configuration => mailet/rate-limiter}/mailetcontainer.xml (95%)
create mode 100644 server/mailet/rate-limiter/src/main/scala/org/apache/james/transport/mailets/PerSenderRateLimit.scala
create mode 100644 server/mailet/rate-limiter/src/test/scala/org/apache/james/transport/mailets/PerSenderRateLimitTest.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 05/05: JAMES-3683 RateLimiter: Allow using longs for allowed quantities
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit e019beaf012841338fd28669613f8b09bd4317d0
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Jan 20 14:09:03 2022 +0700
JAMES-3683 RateLimiter: Allow using longs for allowed quantities
This forces splitting types between increments and allowed quatities...
AllowedQuantity -> long
Increment -> int (the limitation comes from the underlying library)
Having a single mail doing an increment over 2 billion seems unlikely and,
if this is happening, should be rejected.
Also adds unit tests demonstrating that overflow cannot be leveraged to cheat
on the `totalSize` parameter.
---
server/mailet/rate-limiter/README.adoc | 3 +-
.../james/rate/limiter/api/RateLimiter.scala | 32 +++++++++++---
.../rate/limiter/memory/MemoryRateLimiter.scala | 4 +-
.../transport/mailets/PerSenderRateLimit.scala | 43 ++++++++++--------
.../rate/limiter/api/RateLimiterContract.scala | 16 +++----
.../transport/mailets/PerSenderRateLimitTest.scala | 51 ++++++++++++++++++++++
6 files changed, 114 insertions(+), 35 deletions(-)
diff --git a/server/mailet/rate-limiter/README.adoc b/server/mailet/rate-limiter/README.adoc
index 26ee5bd..6253039 100644
--- a/server/mailet/rate-limiter/README.adoc
+++ b/server/mailet/rate-limiter/README.adoc
@@ -43,7 +43,8 @@ Here are supported configuration parameters:
- *count*: Count of emails allowed for a given sender during duration. Optional, if unspecified this rate limit is not applied.
- *recipients*: Count of recipients allowed for a given sender during duration. Optional, if unspecified this rate limit is not applied.
- *size*: Size of emails allowed for a given sender during duration (each email count one time, regardless of recipient count). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B.
- - *totalSize*: Size of emails allowed for a given sender during duration (each recipient of the email email count one time). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B.
+ - *totalSize*: Size of emails allowed for a given sender during duration (each recipient of the email email count one time). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B. Note that
+ totalSize is limited in increments of 2exp(31) - ~2 billions: sending a 10MB file to more than 205 recipients will be rejected if this parameter is enabled.
For instance, to apply all the examples given above:
diff --git a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala
index 6b0ad58..83c69ba 100644
--- a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala
+++ b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala
@@ -24,7 +24,8 @@ import java.time.Duration
import eu.timepit.refined
import eu.timepit.refined.api.Refined
import eu.timepit.refined.numeric.Positive
-import org.apache.james.rate.limiter.api.Quantity.Quantity
+import org.apache.james.rate.limiter.api.AllowedQuantity.AllowedQuantity
+import org.apache.james.rate.limiter.api.Increment.Increment
import org.apache.mailet.MailetConfig
import org.reactivestreams.Publisher
@@ -32,28 +33,45 @@ trait RateLimitingKey {
def asString(): String
}
-object Quantity {
+object AllowedQuantity {
type PositiveLongConstraint = Positive
- type Quantity = Int Refined PositiveLongConstraint
+ type AllowedQuantity = Long Refined PositiveLongConstraint
- def validate(value: Int): Either[NumberFormatException, Quantity] =
+ def validate(value: Long): Either[NumberFormatException, AllowedQuantity] =
refined.refineV[PositiveLongConstraint](value) match {
case Right(value) => Right(value)
case Left(error) => Left(new NumberFormatException(error))
}
- def liftOrThrow(value: Int): Quantity =
+ def liftOrThrow(value: Long): AllowedQuantity =
validate(value) match {
case Right(value) => value
case Left(error) => throw error
}
}
-case class Rule(quantity: Quantity, duration: Duration)
+object Increment {
+ type PositiveLongConstraint = Positive
+ type Increment = Int Refined PositiveLongConstraint
+
+ def validate(value: Int): Either[NumberFormatException, Increment] =
+ refined.refineV[PositiveLongConstraint](value) match {
+ case Right(value) => Right(value)
+ case Left(error) => Left(new NumberFormatException(error))
+ }
+
+ def liftOrThrow(value: Int): Increment =
+ validate(value) match {
+ case Right(value) => value
+ case Left(error) => throw error
+ }
+}
+
+case class Rule(quantity: AllowedQuantity, duration: Duration)
case class Rules(rules: Seq[Rule])
trait RateLimiter {
- def rateLimit(key: RateLimitingKey, increaseQuantity: Quantity): Publisher[RateLimitingResult]
+ def rateLimit(key: RateLimitingKey, increaseQuantity: Increment): Publisher[RateLimitingResult]
}
trait RateLimiterFactory {
diff --git a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/memory/MemoryRateLimiter.scala b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/memory/MemoryRateLimiter.scala
index 7c764cf..c57e9d8 100644
--- a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/memory/MemoryRateLimiter.scala
+++ b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/memory/MemoryRateLimiter.scala
@@ -24,7 +24,7 @@ import java.time.Duration
import com.google.inject.AbstractModule
import es.moki.ratelimitj.core.limiter.request.{RequestLimitRule, RequestRateLimiter}
import es.moki.ratelimitj.inmemory.request.InMemorySlidingWindowRequestRateLimiter
-import org.apache.james.rate.limiter.api.Quantity.Quantity
+import org.apache.james.rate.limiter.api.Increment.Increment
import org.apache.james.rate.limiter.api.{AcceptableRate, RateExceeded, RateLimiter, RateLimiterFactory, RateLimiterFactoryProvider, RateLimitingKey, RateLimitingResult, Rule, Rules}
import org.apache.mailet.MailetConfig
import org.reactivestreams.Publisher
@@ -58,7 +58,7 @@ class MemoryRateLimiterFactory(precision: Option[Duration] = None) extends RateL
}
case class MemoryRateLimiter(limiter: RequestRateLimiter) extends RateLimiter {
- override def rateLimit(key: RateLimitingKey, increaseQuantity: Quantity): Publisher[RateLimitingResult] =
+ override def rateLimit(key: RateLimitingKey, increaseQuantity: Increment): Publisher[RateLimitingResult] =
if (limiter.overLimitWhenIncremented(key.asString(), increaseQuantity.value)) {
SMono.just(RateExceeded)
} else {
diff --git a/server/mailet/rate-limiter/src/main/scala/org/apache/james/transport/mailets/PerSenderRateLimit.scala b/server/mailet/rate-limiter/src/main/scala/org/apache/james/transport/mailets/PerSenderRateLimit.scala
index 89207c2..4124bff 100644
--- a/server/mailet/rate-limiter/src/main/scala/org/apache/james/transport/mailets/PerSenderRateLimit.scala
+++ b/server/mailet/rate-limiter/src/main/scala/org/apache/james/transport/mailets/PerSenderRateLimit.scala
@@ -26,48 +26,56 @@ import com.google.common.annotations.VisibleForTesting
import eu.timepit.refined.auto._
import javax.inject.Inject
import org.apache.james.core.MailAddress
-import org.apache.james.rate.limiter.api.Quantity.Quantity
-import org.apache.james.rate.limiter.api.{AcceptableRate, Quantity, RateExceeded, RateLimiter, RateLimiterFactoryProvider, RateLimitingKey, RateLimitingResult, Rule, Rules}
+import org.apache.james.rate.limiter.api.Increment.Increment
+import org.apache.james.rate.limiter.api.{AcceptableRate, AllowedQuantity, Increment, RateExceeded, RateLimiter, RateLimiterFactoryProvider, RateLimitingKey, RateLimitingResult, Rule, Rules}
import org.apache.james.util.DurationParser
import org.apache.mailet.Mail
import org.apache.mailet.base.GenericMailet
import org.reactivestreams.Publisher
import reactor.core.scala.publisher.{SFlux, SMono}
+import scala.util.Try
+
case class KeyPrefix(value: String)
sealed trait EntityType {
def asString(): String
- def extractQuantity(mail: Mail): Quantity
+ def extractQuantity(mail: Mail): Option[Increment]
}
case object Count extends EntityType {
override def asString(): String = "count"
- override def extractQuantity(mail: Mail): Quantity = 1
+ override def extractQuantity(mail: Mail): Option[Increment] = Some(1)
}
case object RecipientsType extends EntityType {
override def asString(): String = "recipients"
- override def extractQuantity(mail: Mail): Quantity = Quantity.liftOrThrow(mail.getRecipients.size())
+ override def extractQuantity(mail: Mail): Option[Increment] = Some(Increment.liftOrThrow(mail.getRecipients.size()))
}
case object Size extends EntityType {
override def asString(): String = "size"
- override def extractQuantity(mail: Mail): Quantity = Quantity.liftOrThrow(mail.getMessageSize.toInt)
+ override def extractQuantity(mail: Mail): Option[Increment] = Some(Increment.liftOrThrow(mail.getMessageSize.toInt))
}
case object TotalSize extends EntityType {
override def asString(): String = "totalSize"
- override def extractQuantity(mail: Mail): Quantity = Quantity.liftOrThrow((mail.getMessageSize * mail.getRecipients.size()).toInt)
+ override def extractQuantity(mail: Mail): Option[Increment] =
+ Try(Math.multiplyExact(mail.getMessageSize.toInt, mail.getRecipients.size()).toInt)
+ .map(Increment.liftOrThrow)
+ .toOption
}
case class PerSenderRateLimiter(rateLimiter: Option[RateLimiter], keyPrefix: Option[KeyPrefix], entityType: EntityType) {
def rateLimit(sender: MailAddress, mail: Mail): Publisher[RateLimitingResult] = {
val rateLimitingKey = SenderKey(keyPrefix, entityType, sender)
- rateLimiter.map(limiter => limiter.rateLimit(rateLimitingKey, entityType.extractQuantity(mail)))
+ rateLimiter.map(limiter =>
+ entityType.extractQuantity(mail)
+ .map(increment => limiter.rateLimit(rateLimitingKey, increment))
+ .getOrElse(SMono.just[RateLimitingResult](RateExceeded)))
.getOrElse(SMono.just[RateLimitingResult](AcceptableRate))
}
}
@@ -99,7 +107,8 @@ case class SenderKey(keyPrefix: Option[KeyPrefix], entityType: EntityType, mailA
* <li><b>count</b>: Count of emails allowed for a given sender during duration. Optional, if unspecified this rate limit is not applied.</li>
* <li><b>recipients</b>: Count of recipients allowed for a given sender during duration. Optional, if unspecified this rate limit is not applied.</li>
* <li><b>size</b>: Size of emails allowed for a given sender during duration (each email count one time, regardless of recipient count). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B.</li>
- * <li><b>totalSize</b>: Size of emails allowed for a given sender during duration (each recipient of the email email count one time). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B.</li>
+ * <li><b>totalSize</b>: Size of emails allowed for a given sender during duration (each recipient of the email email count one time). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B. Note that
+ * totalSize is limited in increments of 2exp(31) - ~2 billions: sending a 10MB file to more than 205 recipients will be rejected if this parameter is enabled.</li>
* </ul>
*
* <p>For instance, to apply all the examples given above:</p>
@@ -159,13 +168,13 @@ class PerSenderRateLimit @Inject()(rateLimiterFactoryProvider: RateLimiterFactor
.getOrElse(throw new IllegalArgumentException("'duration' is compulsory"))
private def extractCountRules(duration: Duration): Option[Rules] = Option(getInitParameter("count"))
- .map(_.toInt)
- .map(Quantity.liftOrThrow)
+ .map(_.toLong)
+ .map(AllowedQuantity.liftOrThrow)
.map(quantity => Rules(Seq(Rule(quantity, duration))))
private def extractRecipientsRules(duration: Duration): Option[Rules] = Option(getInitParameter("recipients"))
- .map(_.toInt)
- .map(Quantity.liftOrThrow)
+ .map(_.toLong)
+ .map(AllowedQuantity.liftOrThrow)
.map(quantity => Rules(Seq(Rule(quantity, duration))))
private def extractSizeRules(duration: Duration): Option[Rules] = Option(getInitParameter("size"))
@@ -174,8 +183,8 @@ class PerSenderRateLimit @Inject()(rateLimiterFactoryProvider: RateLimiterFactor
case s => s
}
.map(org.apache.james.util.Size.parse)
- .map(_.asBytes().toInt)
- .map(Quantity.liftOrThrow)
+ .map(_.asBytes())
+ .map(AllowedQuantity.liftOrThrow)
.map(quantity => Rules(Seq(Rule(quantity, duration))))
private def extractTotalSizeRules(duration: Duration): Option[Rules] = Option(getInitParameter("totalSize"))
@@ -184,8 +193,8 @@ class PerSenderRateLimit @Inject()(rateLimiterFactoryProvider: RateLimiterFactor
case s => s
}
.map(org.apache.james.util.Size.parse)
- .map(_.asBytes().toInt)
- .map(Quantity.liftOrThrow)
+ .map(_.asBytes())
+ .map(AllowedQuantity.liftOrThrow)
.map(quantity => Rules(Seq(Rule(quantity, duration))))
override def service(mail: Mail): Unit = mail.getMaybeSender
diff --git a/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala b/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala
index 9b7bc44..62227a7 100644
--- a/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala
+++ b/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala
@@ -32,7 +32,7 @@ case class TestKey(value: String) extends RateLimitingKey {
}
trait RateLimiterContract {
- private val rules = Rules(Seq(Rule(4, Duration.ofSeconds(2))))
+ private val rules = Rules(Seq(Rule(4L, Duration.ofSeconds(2))))
def testee(): RateLimiterFactory
@@ -77,7 +77,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsByBlockOverLimitsWithSmallPauseShouldBeExceeded(): Unit = {
- val rateLimiter = testee().withSpecification(Rules(Seq(Rule(4, Duration.ofSeconds(20)))))
+ val rateLimiter = testee().withSpecification(Rules(Seq(Rule(4L, Duration.ofSeconds(20)))))
SMono(rateLimiter.rateLimit(TestKey("key1"), 2)).block()
sleep(Duration.ofSeconds(1))
@@ -107,8 +107,8 @@ trait RateLimiterContract {
@Test
def shouldFailWhenUpperLimitExceeded(): Unit = {
val rateLimiter = testee().withSpecification(Rules(Seq(
- Rule(1, Duration.ofSeconds(1)),
- Rule(2, Duration.ofSeconds(20)))))
+ Rule(1L, Duration.ofSeconds(1)),
+ Rule(2L, Duration.ofSeconds(20)))))
SMono(rateLimiter.rateLimit(TestKey("key1"), 1)).block()
sleep(Duration.ofMillis(1100))
@@ -122,8 +122,8 @@ trait RateLimiterContract {
@Test
def shouldFailWhenLowerLimitExceeded(): Unit = {
val rateLimiter = testee().withSpecification(Rules(Seq(
- Rule(2, Duration.ofSeconds(5)),
- Rule(1, Duration.ofSeconds(1)))))
+ Rule(2L, Duration.ofSeconds(5)),
+ Rule(1L, Duration.ofSeconds(1)))))
SMono(rateLimiter.rateLimit(TestKey("key1"), 1)).block()
// 2 requests in less than 1 s -> fail
@@ -134,8 +134,8 @@ trait RateLimiterContract {
@Test
def shouldSucceedWhenBothRulesAreRespected(): Unit = {
val rateLimiter = testee().withSpecification(Rules(Seq(
- Rule(2, Duration.ofSeconds(5)),
- Rule(1, Duration.ofSeconds(1)))))
+ Rule(2L, Duration.ofSeconds(5)),
+ Rule(1L, Duration.ofSeconds(1)))))
SMono(rateLimiter.rateLimit(TestKey("key1"), 1)).block()
sleep(Duration.ofSeconds(1))
diff --git a/server/mailet/rate-limiter/src/test/scala/org/apache/james/transport/mailets/PerSenderRateLimitTest.scala b/server/mailet/rate-limiter/src/test/scala/org/apache/james/transport/mailets/PerSenderRateLimitTest.scala
index 14f81e0..aa3cce6 100644
--- a/server/mailet/rate-limiter/src/test/scala/org/apache/james/transport/mailets/PerSenderRateLimitTest.scala
+++ b/server/mailet/rate-limiter/src/test/scala/org/apache/james/transport/mailets/PerSenderRateLimitTest.scala
@@ -19,6 +19,7 @@
package org.apache.james.transport.mailets
+import org.apache.james.core.MailAddress
import org.apache.james.rate.limiter.memory.MemoryRateLimiterFactoryProvider
import org.apache.mailet.base.test.{FakeMail, FakeMailetConfig}
import org.apache.mailet.{Mail, MailetConfig}
@@ -26,6 +27,8 @@ import org.assertj.core.api.Assertions.{assertThat, assertThatCode, assertThatTh
import org.assertj.core.api.SoftAssertions
import org.junit.jupiter.api.{Nested, Test}
+import scala.jdk.CollectionConverters._
+
class PerSenderRateLimitTest {
def testee(mailetConfig: MailetConfig): PerSenderRateLimit = {
val mailet = new PerSenderRateLimit(new MemoryRateLimiterFactoryProvider())
@@ -251,6 +254,54 @@ class PerSenderRateLimitTest {
}
@Test
+ def totalSizeShouldRejectWhenOverflowing(): Unit = {
+ val mailet = testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("duration", "20s")
+ .setProperty("totalSize", "1G")
+ .build())
+
+ val recipients = (1 to 450).map(i => new MailAddress(s"rcpt${i}@domain.tld")).toList
+
+ // 450 * 10 MB = 4.5GB (overflow to a positive integer)
+ val mail: Mail = FakeMail.builder()
+ .name("mail1")
+ .sender("sender@domain.tld")
+ .recipients(recipients.asJava)
+ .size(10L * 1024L * 1024L)
+ .state("transport")
+ .build()
+
+ mailet.service(mail)
+
+ assertThat(mail.getState).isEqualTo("error")
+ }
+
+ @Test
+ def overflowOnTotalSizeShouldNotAffectOtherCriteriaWhenUnspecified(): Unit = {
+ val mailet = testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("duration", "20s")
+ .setProperty("count", "2")
+ .build())
+
+ val recipients = (1 to 450).map(i => new MailAddress(s"rcpt${i}@domain.tld")).toList
+
+ // 450 * 10 MB = 4.5GB (overflow to a positive integer)
+ val mail: Mail = FakeMail.builder()
+ .name("mail1")
+ .sender("sender@domain.tld")
+ .recipients(recipients.asJava)
+ .size(10L * 1024L * 1024L)
+ .state("transport")
+ .build()
+
+ mailet.service(mail)
+
+ assertThat(mail.getState).isEqualTo("transport")
+ }
+
+ @Test
def severalLimitsShouldBeApplied(): Unit = {
val mailet = testee(FakeMailetConfig.builder()
.mailetName("PerSenderRateLimit")
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 03/05: JAMES-3683 Docker-compose demonstrating the setup of PerSenderRateLimit mailet
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit b8eb29984523bfa38537567e951d4c2321d11869
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Wed Jan 19 16:47:46 2022 +0700
JAMES-3683 Docker-compose demonstrating the setup of PerSenderRateLimit mailet
---
server/mailet/rate-limiter/README.adoc | 60 ++++++++
server/mailet/rate-limiter/docker-compose.yml | 12 ++
server/mailet/rate-limiter/extensions.properties | 1 +
server/mailet/rate-limiter/mailetcontainer.xml | 170 +++++++++++++++++++++++
4 files changed, 243 insertions(+)
diff --git a/server/mailet/rate-limiter/README.adoc b/server/mailet/rate-limiter/README.adoc
new file mode 100644
index 0000000..26ee5bd
--- /dev/null
+++ b/server/mailet/rate-limiter/README.adoc
@@ -0,0 +1,60 @@
+= Rate limiting for Apache James
+
+This maven module provide a James extension allowing one to enforce rate limiting for Apache James email processing.
+
+== Running rate limiting with the memory implementation
+
+ - 1. Compile this project.
+ - 2. Place the generated "jar with dependencies" in James `extensions-jars` folder.
+ - 3. Configure James to use the memory implementation for Rate Limiting by putting the following content within
+`extension.properties`:
+
+----
+guice.extension.module=org.apache.james.rate.limiter.memory.MemoryRateLimiterModule
+----
+
+ - 4. Use the rate limiting mailets within `mailetcontainer.xml`
+
+We provide a sample `docker-compose` demonstrating this setup.
+
+== Mailets
+
+=== PerSenderRateLimit
+
+*PerSenderRateLimit* allows defining and enforcing rate limits for the sender of matching emails.
+
+This allows writing rules like:
+
+ - A sender can send 10 emails per hour
+ - A sender can send email to a total of 20 recipients per hour
+ - A sender can send 100 MB of emails (total computed taking only the email size into account) per hour
+ - A sender can send 200 MB of emails (total computed taking each recipient copies into account) per hour
+
+Depending on its position and the matcher it is being combined with, those rate limiting rules could be applied to
+submitted emails, received emails or emitted email being relayed to third parties.
+
+Here are supported configuration parameters:
+
+ - *keyPrefix*: An optional key prefix to apply to rate limiting. Choose distinct values if you specify
+ this mailet twice within your `mailetcontainer.xml` file. Defaults to none.
+ - *exceededProcessor*: Processor to which emails whose rate is exceeded should be redirected to. Defaults to error.
+ Use this to customize the behaviour upon exceeded rate.
+ - *duration*: Duration during which the rate limiting shall be applied. Compulsory, must be a valid duration of at least one second. Supported units includes s (second), m (minute), h (hour), d (day).
+ - *count*: Count of emails allowed for a given sender during duration. Optional, if unspecified this rate limit is not applied.
+ - *recipients*: Count of recipients allowed for a given sender during duration. Optional, if unspecified this rate limit is not applied.
+ - *size*: Size of emails allowed for a given sender during duration (each email count one time, regardless of recipient count). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B.
+ - *totalSize*: Size of emails allowed for a given sender during duration (each recipient of the email email count one time). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B.
+
+
+For instance, to apply all the examples given above:
+
+----
+<mailet matcher="All" class="PerSenderRateLimit">
+ <keyPrefix>myPrefix</keyPrefix>
+ <duration>1h</duration>
+ <count>10</count>
+ <recipients>20</recipients>
+ <size>100M</size>
+ <totalSize>200M</totalSize>
+</mailet>
+----
\ No newline at end of file
diff --git a/server/mailet/rate-limiter/docker-compose.yml b/server/mailet/rate-limiter/docker-compose.yml
new file mode 100644
index 0000000..5b4f8c9
--- /dev/null
+++ b/server/mailet/rate-limiter/docker-compose.yml
@@ -0,0 +1,12 @@
+version: '3'
+
+services:
+
+ james:
+ image: apache/james:memory-latest
+ container_name: james
+ hostname: james.local
+ volumes:
+ - $PWD/target/james-server-rate-limiter-3.7.0-SNAPSHOT-jar-with-dependencies.jar:/root/extensions-jars/james-server-rate-limiter.jar
+ - $PWD/extensions.properties:/root/conf/extensions.properties
+ - $PWD/mailetcontainer.xml:/root/conf/mailetcontainer.xml
diff --git a/server/mailet/rate-limiter/extensions.properties b/server/mailet/rate-limiter/extensions.properties
new file mode 100644
index 0000000..d56f517
--- /dev/null
+++ b/server/mailet/rate-limiter/extensions.properties
@@ -0,0 +1 @@
+guice.extension.module=org.apache.james.rate.limiter.memory.MemoryRateLimiterModule
\ No newline at end of file
diff --git a/server/mailet/rate-limiter/mailetcontainer.xml b/server/mailet/rate-limiter/mailetcontainer.xml
new file mode 100644
index 0000000..8804b31
--- /dev/null
+++ b/server/mailet/rate-limiter/mailetcontainer.xml
@@ -0,0 +1,170 @@
+<?xml version="1.0"?>
+
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+ -->
+
+<!-- Read https://james.apache.org/server/config-mailetcontainer.html for further details -->
+
+<mailetcontainer enableJmx="true">
+
+ <context>
+ <!-- When the domain part of the postmaster mailAddress is missing, the default domain is appended.
+ You can configure it to (for example) <postmaster>postmaster@myDomain.com</postmaster> -->
+ <postmaster>postmaster</postmaster>
+ </context>
+
+ <spooler>
+ <threads>20</threads>
+ <errorRepository>memory://var/mail/error/</errorRepository>
+ </spooler>
+
+ <processors>
+ <processor state="root" enableJmx="true">
+ <mailet match="All" class="PostmasterAlias"/>
+ <mailet match="RelayLimit=30" class="Null"/>
+ <mailet match="All" class="ToProcessor">
+ <processor>transport</processor>
+ </mailet>
+ </processor>
+
+ <processor state="error" enableJmx="true">
+ <mailet match="All" class="MetricsMailet">
+ <metricName>mailetContainerErrors</metricName>
+ </mailet>
+ <mailet match="All" class="Bounce">
+ <onMailetException>ignore</onMailetException>
+ </mailet>
+ <mailet match="All" class="ToRepository">
+ <repositoryPath>memory://var/mail/error/</repositoryPath>
+ <onMailetException>propagate</onMailetException>
+ </mailet>
+ </processor>
+
+ <processor state="transport" enableJmx="true">
+ <matcher name="relay-allowed" match="org.apache.james.mailetcontainer.impl.matchers.Or">
+ <matcher match="SMTPAuthSuccessful"/>
+ <matcher match="SMTPIsAuthNetwork"/>
+ <matcher match="SentByMailet"/>
+ <matcher match="org.apache.james.jmap.mailet.SentByJmap"/>
+ </matcher>
+
+ <mailet match="All" class="RemoveMimeHeader">
+ <name>bcc</name>
+ <onMailetException>ignore</onMailetException>
+ </mailet>
+ <mailet match="All" class="RecipientRewriteTable">
+ <errorProcessor>rrt-error</errorProcessor>
+ </mailet>
+ <mailet match="RecipientIsLocal" class="ToProcessor">
+ <processor>local-delivery</processor>
+ </mailet>
+ <mailet match="HostIsLocal" class="ToProcessor">
+ <processor>local-address-error</processor>
+ <notice>550 - Requested action not taken: no such user here</notice>
+ </mailet>
+ <mailet match="relay-allowed" class="ToProcessor">
+ <processor>relay</processor>
+ </mailet>
+ <mailet match="All" class="ToProcessor">
+ <processor>relay-denied</processor>
+ </mailet>
+ </processor>
+
+ <processor state="local-delivery" enableJmx="true">
+ <mailet match="All" class="VacationMailet">
+ <onMailetException>ignore</onMailetException>
+ </mailet>
+ <mailet match="All" class="Sieve">
+ <onMailetException>ignore</onMailetException>
+ </mailet>
+ <mailet match="All" class="AddDeliveredToHeader"/>
+ <mailet match="All" class="org.apache.james.jmap.mailet.filter.JMAPFiltering">
+ <onMailetException>ignore</onMailetException>
+ </mailet>
+ <mailet match="All" class="LocalDelivery"/>
+ </processor>
+
+ <processor state="relay" enableJmx="true">
+ <mailet matcher="All" class="PerSenderRateLimit">
+ <keyPrefix>myPrefix</keyPrefix>
+ <duration>1h</duration>
+ <count>10</count>
+ <recipients>20</recipients>
+ <size>100M</size>
+ <totalSize>200M</totalSize>
+ </mailet>
+ <mailet match="All" class="RemoteDelivery">
+ <outgoingQueue>outgoing</outgoingQueue>
+ <delayTime>5000, 100000, 500000</delayTime>
+ <maxRetries>3</maxRetries>
+ <maxDnsProblemRetries>0</maxDnsProblemRetries>
+ <deliveryThreads>10</deliveryThreads>
+ <sendpartial>true</sendpartial>
+ <bounceProcessor>bounces</bounceProcessor>
+ </mailet>
+ </processor>
+
+ <processor state="local-address-error" enableJmx="true">
+ <mailet match="All" class="MetricsMailet">
+ <metricName>mailetContainerLocalAddressError</metricName>
+ </mailet>
+ <mailet match="All" class="Bounce">
+ <attachment>none</attachment>
+ </mailet>
+ <mailet match="All" class="ToRepository">
+ <repositoryPath>memory://var/mail/address-error/</repositoryPath>
+ </mailet>
+ </processor>
+
+ <processor state="relay-denied" enableJmx="true">
+ <mailet match="All" class="MetricsMailet">
+ <metricName>mailetContainerRelayDenied</metricName>
+ </mailet>
+ <mailet match="All" class="Bounce">
+ <attachment>none</attachment>
+ </mailet>
+ <mailet match="All" class="ToRepository">
+ <repositoryPath>memory://var/mail/relay-denied/</repositoryPath>
+ <notice>Warning: You are sending an e-mail to a remote server. You must be authenticated to perform such an operation</notice>
+ </mailet>
+ </processor>
+
+ <processor state="bounces" enableJmx="true">
+ <mailet match="All" class="MetricsMailet">
+ <metricName>bounces</metricName>
+ </mailet>
+ <mailet match="All" class="DSNBounce">
+ <passThrough>false</passThrough>
+ </mailet>
+ </processor>
+
+ <processor state="rrt-error" enableJmx="false">
+ <mailet match="All" class="ToRepository">
+ <repositoryPath>memory://var/mail/rrt-error/</repositoryPath>
+ <passThrough>true</passThrough>
+ </mailet>
+ <mailet match="IsSenderInRRTLoop" class="Null"/>
+ <mailet match="All" class="Bounce"/>
+ </processor>
+
+ </processors>
+
+</mailetcontainer>
+
+
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 01/05: JAMES-3693 Allow instanciation of MemoryRateLimiter from within mailets
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 29fd7c69d7c05e4cd8796d73f6ffaa3cee7baf0c
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Wed Jan 19 14:49:04 2022 +0700
JAMES-3693 Allow instanciation of MemoryRateLimiter from within mailets
---
server/mailet/rate-limiter/pom.xml | 8 ++++++++
.../apache/james/rate/limiter/api/RateLimiter.scala | 5 +++++
.../rate/limiter/memory/MemoryRateLimiter.scala | 14 +++++++++++++-
.../james/rate/limiter/api/RateLimiterContract.scala | 20 ++++++++++----------
4 files changed, 36 insertions(+), 11 deletions(-)
diff --git a/server/mailet/rate-limiter/pom.xml b/server/mailet/rate-limiter/pom.xml
index 2b9b2e9..4bf8d11 100644
--- a/server/mailet/rate-limiter/pom.xml
+++ b/server/mailet/rate-limiter/pom.xml
@@ -36,10 +36,18 @@
<dependencies>
<dependency>
<groupId>${james.groupId}</groupId>
+ <artifactId>apache-mailet-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>${james.groupId}</groupId>
<artifactId>james-server-testing</artifactId>
<scope>test</scope>
</dependency>
<dependency>
+ <groupId>com.google.inject</groupId>
+ <artifactId>guice</artifactId>
+ </dependency>
+ <dependency>
<groupId>es.moki.ratelimitj</groupId>
<artifactId>ratelimitj-inmemory</artifactId>
<version>0.7.0</version>
diff --git a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala
index 0f61d77..06747b4 100644
--- a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala
+++ b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala
@@ -25,6 +25,7 @@ import eu.timepit.refined
import eu.timepit.refined.api.Refined
import eu.timepit.refined.numeric.Positive
import org.apache.james.rate.limiter.api.Quantity.Quantity
+import org.apache.mailet.MailetConfig
import org.reactivestreams.Publisher
trait RateLimitingKey {
@@ -59,6 +60,10 @@ trait RateLimiterFactory {
def withSpecification(rules: Rules): RateLimiter
}
+trait RateLimiterFactoryProvider {
+ def create(mailetConfig: MailetConfig): RateLimiterFactory
+}
+
sealed trait RateLimitingResult
case object RateExceeded extends RateLimitingResult
case object AcceptableRate extends RateLimitingResult
\ No newline at end of file
diff --git a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/memory/MemoryRateLimiter.scala b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/memory/MemoryRateLimiter.scala
index ed93c95..7c764cf 100644
--- a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/memory/MemoryRateLimiter.scala
+++ b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/memory/MemoryRateLimiter.scala
@@ -21,15 +21,27 @@ package org.apache.james.rate.limiter.memory
import java.time.Duration
+import com.google.inject.AbstractModule
import es.moki.ratelimitj.core.limiter.request.{RequestLimitRule, RequestRateLimiter}
import es.moki.ratelimitj.inmemory.request.InMemorySlidingWindowRequestRateLimiter
import org.apache.james.rate.limiter.api.Quantity.Quantity
-import org.apache.james.rate.limiter.api.{AcceptableRate, RateExceeded, RateLimiter, RateLimiterFactory, RateLimitingKey, RateLimitingResult, Rule, Rules}
+import org.apache.james.rate.limiter.api.{AcceptableRate, RateExceeded, RateLimiter, RateLimiterFactory, RateLimiterFactoryProvider, RateLimitingKey, RateLimitingResult, Rule, Rules}
+import org.apache.mailet.MailetConfig
import org.reactivestreams.Publisher
import reactor.core.scala.publisher.SMono
import scala.jdk.CollectionConverters._
+class MemoryRateLimiterModule() extends AbstractModule {
+ override def configure(): Unit =
+ bind(classOf[RateLimiterFactoryProvider])
+ .to(classOf[MemoryRateLimiterFactoryProvider])
+}
+
+class MemoryRateLimiterFactoryProvider() extends RateLimiterFactoryProvider {
+ override def create(mailetConfig: MailetConfig): RateLimiterFactory = new MemoryRateLimiterFactory()
+}
+
class MemoryRateLimiterFactory(precision: Option[Duration] = None) extends RateLimiterFactory {
override def withSpecification(rules: Rules): RateLimiter = {
MemoryRateLimiter(new InMemorySlidingWindowRequestRateLimiter(rules.rules
diff --git a/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala b/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala
index 9b7bc44..146e754 100644
--- a/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala
+++ b/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala
@@ -40,7 +40,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsBelowLimitsWithoutPauseShouldBeAcceptable(): Unit = {
- val rateLimiter = testee().withSpecification(rules)
+ val rateLimiter = testee().create(rules)
SoftAssertions.assertSoftly(softly => {
(1 to 4).foreach(_ => {
@@ -52,7 +52,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsAtLimitsWithoutPauseShouldBeAcceptable(): Unit = {
- val rateLimiter = testee().withSpecification(rules)
+ val rateLimiter = testee().create(rules)
val actual: RateLimitingResult = SMono(rateLimiter.rateLimit(TestKey("key1"), 4)).block()
assertThat(actual).isEqualTo(AcceptableRate)
@@ -60,7 +60,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsOverLimitsWithoutPauseShouldBeExceeded(): Unit = {
- val rateLimiter = testee().withSpecification(rules)
+ val rateLimiter = testee().create(rules)
val actual: RateLimitingResult = SMono(rateLimiter.rateLimit(TestKey("key1"), 5)).block()
assertThat(actual).isEqualTo(RateExceeded)
@@ -68,7 +68,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsByBlockOverLimitsWithoutPauseShouldBeExceeded(): Unit = {
- val rateLimiter = testee().withSpecification(rules)
+ val rateLimiter = testee().create(rules)
SMono(rateLimiter.rateLimit(TestKey("key1"), 2)).block()
val actual: RateLimitingResult = SMono(rateLimiter.rateLimit(TestKey("key1"), 3)).block()
@@ -77,7 +77,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsByBlockOverLimitsWithSmallPauseShouldBeExceeded(): Unit = {
- val rateLimiter = testee().withSpecification(Rules(Seq(Rule(4, Duration.ofSeconds(20)))))
+ val rateLimiter = testee().create(Rules(Seq(Rule(4, Duration.ofSeconds(20)))))
SMono(rateLimiter.rateLimit(TestKey("key1"), 2)).block()
sleep(Duration.ofSeconds(1))
@@ -87,7 +87,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsByBlockOverLimitsWithLongPauseShouldBeAcceptable(): Unit = {
- val rateLimiter = testee().withSpecification(rules)
+ val rateLimiter = testee().create(rules)
SMono(rateLimiter.rateLimit(TestKey("key1"), 2)).block()
sleep(Duration.ofSeconds(3))
@@ -97,7 +97,7 @@ trait RateLimiterContract {
@Test
def rateLimitingShouldBePartitionedByKey(): Unit = {
- val rateLimiter = testee().withSpecification(rules)
+ val rateLimiter = testee().create(rules)
SMono(rateLimiter.rateLimit(TestKey("key1"), 2)).block()
val actual: RateLimitingResult = SMono(rateLimiter.rateLimit(TestKey("key2"), 3)).block()
@@ -106,7 +106,7 @@ trait RateLimiterContract {
@Test
def shouldFailWhenUpperLimitExceeded(): Unit = {
- val rateLimiter = testee().withSpecification(Rules(Seq(
+ val rateLimiter = testee().create(Rules(Seq(
Rule(1, Duration.ofSeconds(1)),
Rule(2, Duration.ofSeconds(20)))))
@@ -121,7 +121,7 @@ trait RateLimiterContract {
@Test
def shouldFailWhenLowerLimitExceeded(): Unit = {
- val rateLimiter = testee().withSpecification(Rules(Seq(
+ val rateLimiter = testee().create(Rules(Seq(
Rule(2, Duration.ofSeconds(5)),
Rule(1, Duration.ofSeconds(1)))))
@@ -133,7 +133,7 @@ trait RateLimiterContract {
@Test
def shouldSucceedWhenBothRulesAreRespected(): Unit = {
- val rateLimiter = testee().withSpecification(Rules(Seq(
+ val rateLimiter = testee().create(Rules(Seq(
Rule(2, Duration.ofSeconds(5)),
Rule(1, Duration.ofSeconds(1)))))
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 02/05: JAMES-3683 Implement PerSenderRateLimit mailet
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 22cc8711a8121128109bc1471af822a61f6d44cd
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Wed Jan 19 16:47:26 2022 +0700
JAMES-3683 Implement PerSenderRateLimit mailet
-> JavaDoc for documenting its usage
---
server/mailet/rate-limiter/pom.xml | 20 +
.../james/rate/limiter/api/RateLimiter.scala | 12 +-
.../transport/mailets/PerSenderRateLimit.scala | 211 ++++++++
.../rate/limiter/api/RateLimiterContract.scala | 20 +-
.../transport/mailets/PerSenderRateLimitTest.scala | 573 +++++++++++++++++++++
5 files changed, 823 insertions(+), 13 deletions(-)
diff --git a/server/mailet/rate-limiter/pom.xml b/server/mailet/rate-limiter/pom.xml
index 4bf8d11..6c78008 100644
--- a/server/mailet/rate-limiter/pom.xml
+++ b/server/mailet/rate-limiter/pom.xml
@@ -40,6 +40,10 @@
</dependency>
<dependency>
<groupId>${james.groupId}</groupId>
+ <artifactId>apache-mailet-base</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>${james.groupId}</groupId>
<artifactId>james-server-testing</artifactId>
<scope>test</scope>
</dependency>
@@ -88,6 +92,22 @@
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
</plugin>
+ <plugin>
+ <artifactId>maven-assembly-plugin</artifactId>
+ <configuration>
+ <descriptorRefs>
+ <descriptorRef>jar-with-dependencies</descriptorRef>
+ </descriptorRefs>
+ </configuration>
+ <executions>
+ <execution>
+ <goals>
+ <goal>single</goal>
+ </goals>
+ <phase>compile</phase>
+ </execution>
+ </executions>
+ </plugin>
</plugins>
</build>
</project>
\ No newline at end of file
diff --git a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala
index 06747b4..6b0ad58 100644
--- a/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala
+++ b/server/mailet/rate-limiter/src/main/scala/org/apache/james/rate/limiter/api/RateLimiter.scala
@@ -64,6 +64,12 @@ trait RateLimiterFactoryProvider {
def create(mailetConfig: MailetConfig): RateLimiterFactory
}
-sealed trait RateLimitingResult
-case object RateExceeded extends RateLimitingResult
-case object AcceptableRate extends RateLimitingResult
\ No newline at end of file
+sealed trait RateLimitingResult {
+ def merge(other: RateLimitingResult): RateLimitingResult
+}
+case object RateExceeded extends RateLimitingResult {
+ override def merge(other: RateLimitingResult): RateLimitingResult = RateExceeded
+}
+case object AcceptableRate extends RateLimitingResult {
+ override def merge(other: RateLimitingResult): RateLimitingResult = other
+}
\ No newline at end of file
diff --git a/server/mailet/rate-limiter/src/main/scala/org/apache/james/transport/mailets/PerSenderRateLimit.scala b/server/mailet/rate-limiter/src/main/scala/org/apache/james/transport/mailets/PerSenderRateLimit.scala
new file mode 100644
index 0000000..89207c2
--- /dev/null
+++ b/server/mailet/rate-limiter/src/main/scala/org/apache/james/transport/mailets/PerSenderRateLimit.scala
@@ -0,0 +1,211 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.transport.mailets
+
+import java.time.Duration
+import java.time.temporal.ChronoUnit
+
+import com.google.common.annotations.VisibleForTesting
+import eu.timepit.refined.auto._
+import javax.inject.Inject
+import org.apache.james.core.MailAddress
+import org.apache.james.rate.limiter.api.Quantity.Quantity
+import org.apache.james.rate.limiter.api.{AcceptableRate, Quantity, RateExceeded, RateLimiter, RateLimiterFactoryProvider, RateLimitingKey, RateLimitingResult, Rule, Rules}
+import org.apache.james.util.DurationParser
+import org.apache.mailet.Mail
+import org.apache.mailet.base.GenericMailet
+import org.reactivestreams.Publisher
+import reactor.core.scala.publisher.{SFlux, SMono}
+
+case class KeyPrefix(value: String)
+
+sealed trait EntityType {
+ def asString(): String
+
+ def extractQuantity(mail: Mail): Quantity
+}
+
+case object Count extends EntityType {
+ override def asString(): String = "count"
+
+ override def extractQuantity(mail: Mail): Quantity = 1
+}
+case object RecipientsType extends EntityType {
+ override def asString(): String = "recipients"
+
+ override def extractQuantity(mail: Mail): Quantity = Quantity.liftOrThrow(mail.getRecipients.size())
+}
+case object Size extends EntityType {
+ override def asString(): String = "size"
+
+ override def extractQuantity(mail: Mail): Quantity = Quantity.liftOrThrow(mail.getMessageSize.toInt)
+}
+case object TotalSize extends EntityType {
+ override def asString(): String = "totalSize"
+
+ override def extractQuantity(mail: Mail): Quantity = Quantity.liftOrThrow((mail.getMessageSize * mail.getRecipients.size()).toInt)
+}
+
+case class PerSenderRateLimiter(rateLimiter: Option[RateLimiter], keyPrefix: Option[KeyPrefix], entityType: EntityType) {
+ def rateLimit(sender: MailAddress, mail: Mail): Publisher[RateLimitingResult] = {
+ val rateLimitingKey = SenderKey(keyPrefix, entityType, sender)
+
+ rateLimiter.map(limiter => limiter.rateLimit(rateLimitingKey, entityType.extractQuantity(mail)))
+ .getOrElse(SMono.just[RateLimitingResult](AcceptableRate))
+ }
+}
+
+case class SenderKey(keyPrefix: Option[KeyPrefix], entityType: EntityType, mailAddress: MailAddress) extends RateLimitingKey {
+ override def asString(): String = keyPrefix.map(prefix => s"${prefix.value}_${entityType.asString()}_${mailAddress.asString()}")
+ .getOrElse(s"${entityType.asString()}_${mailAddress.asString()}")
+}
+
+/**
+ * <p><b>PerSenderRateLimit</b> allows defining and enforcing rate limits for the sender of matching emails.</p>
+ *
+ * <ul>This allows writing rules like:
+ * <li>A sender can send 10 emails per hour</li>
+ * <li>A sender can send email to a total of 20 recipients per hour</li>
+ * <li>A sender can send 100 MB of emails (total computed taking only the email size into account) per hour</li>
+ * <li>A sender can send 200 MB of emails (total computed taking each recipient copies into account) per hour</li>
+ * </ul>
+ *
+ * <p>Depending on its position and the matcher it is being combined with, those rate limiting rules could be applied to
+ * submitted emails, received emails or emitted email being relayed to third parties.</p>
+ *
+ * <ul>Here are supported configuration parameters:
+ * <li><b>keyPrefix</b>: An optional key prefix to apply to rate limiting. Choose distinct values if you specify
+ * this mailet twice within your <code>mailetcontainer.xml</code> file. Defaults to none.</li>
+ * <li><b>exceededProcessor</b>: Processor to which emails whose rate is exceeded should be redirected to. Defaults to error.
+ * Use this to customize the behaviour upon exceeded rate.</li>
+ * <li><b>duration</b>: Duration during which the rate limiting shall be applied. Compulsory, must be a valid duration of at least one second. Supported units includes s (second), m (minute), h (hour), d (day).</li>
+ * <li><b>count</b>: Count of emails allowed for a given sender during duration. Optional, if unspecified this rate limit is not applied.</li>
+ * <li><b>recipients</b>: Count of recipients allowed for a given sender during duration. Optional, if unspecified this rate limit is not applied.</li>
+ * <li><b>size</b>: Size of emails allowed for a given sender during duration (each email count one time, regardless of recipient count). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B.</li>
+ * <li><b>totalSize</b>: Size of emails allowed for a given sender during duration (each recipient of the email email count one time). Optional, if unspecified this rate limit is not applied. Supported units : B ( 2^0 ), K ( 2^10 ), M ( 2^20 ), G ( 2^30 ), defaults to B.</li>
+ * </ul>
+ *
+ * <p>For instance, to apply all the examples given above:</p>
+ *
+ * <pre><code>
+ * <mailet matcher="All" class="PerSenderRateLimit">
+ * <keyPrefix>myPrefix</keyPrefix>
+ * <duration>1h</duration>
+ * <count>10</count>
+ * <recipients>20</recipients>
+ * <size>100M</size>
+ * <totalSize>200M</totalSize>
+ * </mailet>
+ * </code></pre>
+ *
+ * <p>Note that to use this extension you need to place the rate-limiter JAR in the <code>extensions-jars</code> folder
+ * and need to configure a viable option to invoke <code>RateLimiterFactoryProvider</code> which can be done by
+ * loading <code>org.apache.james.rate.limiter.memory.MemoryRateLimiterModule</code> Guice module within the
+ * <code>guice.extension.module</code> in <code>extensions.properties</code> configuration file. Note that other Rate
+ * limiter implementation might require extra configuration parameters within your mailet.</p>
+ *
+ * @param rateLimiterFactoryProvider Allows instantiations of the underlying rate limiters.
+ */
+class PerSenderRateLimit @Inject()(rateLimiterFactoryProvider: RateLimiterFactoryProvider) extends GenericMailet {
+ private var countRateLimiter: PerSenderRateLimiter = _
+ private var recipientsRateLimiter: PerSenderRateLimiter = _
+ private var sizeRateLimiter: PerSenderRateLimiter = _
+ private var totalSizeRateLimiter: PerSenderRateLimiter = _
+ private var exceededProcessor: String = _
+ private var keyPrefix: Option[KeyPrefix] = _
+
+ override def init(): Unit = {
+ val rateLimiterFactory = rateLimiterFactoryProvider.create(getMailetConfig)
+ val duration: Duration = parseDuration()
+
+ keyPrefix = Option(getInitParameter("keyPrefix")).map(KeyPrefix)
+ exceededProcessor = getInitParameter("exceededProcessor", Mail.ERROR)
+
+ countRateLimiter = PerSenderRateLimiter(rateLimiter = extractCountRules(duration).map(rateLimiterFactory.withSpecification),
+ keyPrefix = keyPrefix,
+ entityType = Count)
+ recipientsRateLimiter = PerSenderRateLimiter(rateLimiter = extractRecipientsRules(duration).map(rateLimiterFactory.withSpecification),
+ keyPrefix = keyPrefix,
+ entityType = RecipientsType)
+ sizeRateLimiter = PerSenderRateLimiter(rateLimiter = extractSizeRules(duration).map(rateLimiterFactory.withSpecification),
+ keyPrefix = keyPrefix,
+ entityType = Size)
+ totalSizeRateLimiter = PerSenderRateLimiter(rateLimiter = extractTotalSizeRules(duration).map(rateLimiterFactory.withSpecification),
+ keyPrefix = keyPrefix,
+ entityType = TotalSize)
+ }
+
+ @VisibleForTesting
+ def parseDuration(): Duration =
+ Option(getInitParameter("duration"))
+ .map(string => DurationParser.parse(string, ChronoUnit.SECONDS))
+ .getOrElse(throw new IllegalArgumentException("'duration' is compulsory"))
+
+ private def extractCountRules(duration: Duration): Option[Rules] = Option(getInitParameter("count"))
+ .map(_.toInt)
+ .map(Quantity.liftOrThrow)
+ .map(quantity => Rules(Seq(Rule(quantity, duration))))
+
+ private def extractRecipientsRules(duration: Duration): Option[Rules] = Option(getInitParameter("recipients"))
+ .map(_.toInt)
+ .map(Quantity.liftOrThrow)
+ .map(quantity => Rules(Seq(Rule(quantity, duration))))
+
+ private def extractSizeRules(duration: Duration): Option[Rules] = Option(getInitParameter("size"))
+ .map {
+ case "" => throw new IllegalArgumentException("'size' field cannot be empty if specified")
+ case s => s
+ }
+ .map(org.apache.james.util.Size.parse)
+ .map(_.asBytes().toInt)
+ .map(Quantity.liftOrThrow)
+ .map(quantity => Rules(Seq(Rule(quantity, duration))))
+
+ private def extractTotalSizeRules(duration: Duration): Option[Rules] = Option(getInitParameter("totalSize"))
+ .map {
+ case "" => throw new IllegalArgumentException("'totalSize' field cannot be empty if specified")
+ case s => s
+ }
+ .map(org.apache.james.util.Size.parse)
+ .map(_.asBytes().toInt)
+ .map(Quantity.liftOrThrow)
+ .map(quantity => Rules(Seq(Rule(quantity, duration))))
+
+ override def service(mail: Mail): Unit = mail.getMaybeSender
+ .asOptional()
+ .ifPresent(sender => {
+ applyRateLimiter(mail, sender)
+ })
+
+ private def applyRateLimiter(mail: Mail, sender: MailAddress): Unit = {
+ val pivot: RateLimitingResult = AcceptableRate
+ val result = SFlux.merge(Seq(
+ countRateLimiter.rateLimit(sender, mail),
+ recipientsRateLimiter.rateLimit(sender, mail),
+ sizeRateLimiter.rateLimit(sender, mail),
+ totalSizeRateLimiter.rateLimit(sender, mail)))
+ .fold(pivot)((a, b) => a.merge(b))
+ .block()
+
+ if (result.equals(RateExceeded)) {
+ mail.setState(exceededProcessor)
+ }
+ }
+}
diff --git a/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala b/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala
index 146e754..9b7bc44 100644
--- a/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala
+++ b/server/mailet/rate-limiter/src/test/scala/org/apache/james/rate/limiter/api/RateLimiterContract.scala
@@ -40,7 +40,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsBelowLimitsWithoutPauseShouldBeAcceptable(): Unit = {
- val rateLimiter = testee().create(rules)
+ val rateLimiter = testee().withSpecification(rules)
SoftAssertions.assertSoftly(softly => {
(1 to 4).foreach(_ => {
@@ -52,7 +52,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsAtLimitsWithoutPauseShouldBeAcceptable(): Unit = {
- val rateLimiter = testee().create(rules)
+ val rateLimiter = testee().withSpecification(rules)
val actual: RateLimitingResult = SMono(rateLimiter.rateLimit(TestKey("key1"), 4)).block()
assertThat(actual).isEqualTo(AcceptableRate)
@@ -60,7 +60,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsOverLimitsWithoutPauseShouldBeExceeded(): Unit = {
- val rateLimiter = testee().create(rules)
+ val rateLimiter = testee().withSpecification(rules)
val actual: RateLimitingResult = SMono(rateLimiter.rateLimit(TestKey("key1"), 5)).block()
assertThat(actual).isEqualTo(RateExceeded)
@@ -68,7 +68,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsByBlockOverLimitsWithoutPauseShouldBeExceeded(): Unit = {
- val rateLimiter = testee().create(rules)
+ val rateLimiter = testee().withSpecification(rules)
SMono(rateLimiter.rateLimit(TestKey("key1"), 2)).block()
val actual: RateLimitingResult = SMono(rateLimiter.rateLimit(TestKey("key1"), 3)).block()
@@ -77,7 +77,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsByBlockOverLimitsWithSmallPauseShouldBeExceeded(): Unit = {
- val rateLimiter = testee().create(Rules(Seq(Rule(4, Duration.ofSeconds(20)))))
+ val rateLimiter = testee().withSpecification(Rules(Seq(Rule(4, Duration.ofSeconds(20)))))
SMono(rateLimiter.rateLimit(TestKey("key1"), 2)).block()
sleep(Duration.ofSeconds(1))
@@ -87,7 +87,7 @@ trait RateLimiterContract {
@Test
def subsequentRequestsByBlockOverLimitsWithLongPauseShouldBeAcceptable(): Unit = {
- val rateLimiter = testee().create(rules)
+ val rateLimiter = testee().withSpecification(rules)
SMono(rateLimiter.rateLimit(TestKey("key1"), 2)).block()
sleep(Duration.ofSeconds(3))
@@ -97,7 +97,7 @@ trait RateLimiterContract {
@Test
def rateLimitingShouldBePartitionedByKey(): Unit = {
- val rateLimiter = testee().create(rules)
+ val rateLimiter = testee().withSpecification(rules)
SMono(rateLimiter.rateLimit(TestKey("key1"), 2)).block()
val actual: RateLimitingResult = SMono(rateLimiter.rateLimit(TestKey("key2"), 3)).block()
@@ -106,7 +106,7 @@ trait RateLimiterContract {
@Test
def shouldFailWhenUpperLimitExceeded(): Unit = {
- val rateLimiter = testee().create(Rules(Seq(
+ val rateLimiter = testee().withSpecification(Rules(Seq(
Rule(1, Duration.ofSeconds(1)),
Rule(2, Duration.ofSeconds(20)))))
@@ -121,7 +121,7 @@ trait RateLimiterContract {
@Test
def shouldFailWhenLowerLimitExceeded(): Unit = {
- val rateLimiter = testee().create(Rules(Seq(
+ val rateLimiter = testee().withSpecification(Rules(Seq(
Rule(2, Duration.ofSeconds(5)),
Rule(1, Duration.ofSeconds(1)))))
@@ -133,7 +133,7 @@ trait RateLimiterContract {
@Test
def shouldSucceedWhenBothRulesAreRespected(): Unit = {
- val rateLimiter = testee().create(Rules(Seq(
+ val rateLimiter = testee().withSpecification(Rules(Seq(
Rule(2, Duration.ofSeconds(5)),
Rule(1, Duration.ofSeconds(1)))))
diff --git a/server/mailet/rate-limiter/src/test/scala/org/apache/james/transport/mailets/PerSenderRateLimitTest.scala b/server/mailet/rate-limiter/src/test/scala/org/apache/james/transport/mailets/PerSenderRateLimitTest.scala
new file mode 100644
index 0000000..14f81e0
--- /dev/null
+++ b/server/mailet/rate-limiter/src/test/scala/org/apache/james/transport/mailets/PerSenderRateLimitTest.scala
@@ -0,0 +1,573 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.transport.mailets
+
+import org.apache.james.rate.limiter.memory.MemoryRateLimiterFactoryProvider
+import org.apache.mailet.base.test.{FakeMail, FakeMailetConfig}
+import org.apache.mailet.{Mail, MailetConfig}
+import org.assertj.core.api.Assertions.{assertThat, assertThatCode, assertThatThrownBy}
+import org.assertj.core.api.SoftAssertions
+import org.junit.jupiter.api.{Nested, Test}
+
+class PerSenderRateLimitTest {
+ def testee(mailetConfig: MailetConfig): PerSenderRateLimit = {
+ val mailet = new PerSenderRateLimit(new MemoryRateLimiterFactoryProvider())
+ mailet.init(mailetConfig)
+ mailet
+ }
+
+ @Test
+ def rateLimitingShouldBeAppliedPerSender(): Unit = {
+ val mailet = testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("duration", "20s")
+ .setProperty("count", "1")
+ .build())
+
+ val mail1: Mail = FakeMail.builder()
+ .name("mail1")
+ .sender("sender1@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt2@linagora.com")
+ .state("transport")
+ .build()
+
+ val mail2: Mail = FakeMail.builder()
+ .name("mail2")
+ .sender("sender2@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt3@linagora.com", "rcpt4@linagora.com")
+ .state("transport")
+ .build()
+
+ mailet.service(mail1)
+ mailet.service(mail2)
+
+ SoftAssertions.assertSoftly(softly => {
+ softly.assertThat(mail1.getState).isEqualTo("transport")
+ softly.assertThat(mail2.getState).isEqualTo("transport")
+ })
+ }
+
+ @Test
+ def rateLimitedEmailsShouldFlowToTheIntendedProcessor(): Unit = {
+ val mailet = testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("duration", "20s")
+ .setProperty("count", "1")
+ .setProperty("exceededProcessor", "tooMuchMails")
+ .build())
+
+ val mail1: Mail = FakeMail.builder()
+ .name("mail1")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt2@linagora.com")
+ .state("transport")
+ .build()
+
+ val mail2: Mail = FakeMail.builder()
+ .name("mail2")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt3@linagora.com", "rcpt4@linagora.com")
+ .state("transport")
+ .build()
+
+ mailet.service(mail1)
+ mailet.service(mail2)
+
+ SoftAssertions.assertSoftly(softly => {
+ softly.assertThat(mail1.getState).isEqualTo("transport")
+ softly.assertThat(mail2.getState).isEqualTo("tooMuchMails")
+ })
+ }
+
+ @Test
+ def shouldRateLimitCountOfEmails(): Unit = {
+ val mailet = testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("duration", "20s")
+ .setProperty("count", "1")
+ .build())
+
+ val mail1: Mail = FakeMail.builder()
+ .name("mail1")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt2@linagora.com")
+ .state("transport")
+ .build()
+
+ val mail2: Mail = FakeMail.builder()
+ .name("mail2")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt3@linagora.com", "rcpt4@linagora.com")
+ .state("transport")
+ .build()
+
+ mailet.service(mail1)
+ mailet.service(mail2)
+
+ SoftAssertions.assertSoftly(softly => {
+ softly.assertThat(mail1.getState).isEqualTo("transport")
+ softly.assertThat(mail2.getState).isEqualTo("error")
+ })
+ }
+
+ @Test
+ def shouldRateLimitRecipientsOfEmails(): Unit = {
+ val mailet = testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("duration", "20s")
+ .setProperty("recipients", "4")
+ .build())
+
+ val mail1: Mail = FakeMail.builder()
+ .name("mail1")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt2@linagora.com")
+ .state("transport")
+ .build()
+
+ val mail2: Mail = FakeMail.builder()
+ .name("mail2")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt3@linagora.com", "rcpt4@linagora.com")
+ .state("transport")
+ .build()
+
+ val mail3: Mail = FakeMail.builder()
+ .name("mail3")
+ .sender("sender@domain.tld")
+ .recipients("rcpt5@linagora.com")
+ .state("transport")
+ .build()
+
+ mailet.service(mail1)
+ mailet.service(mail2)
+ mailet.service(mail3)
+
+ SoftAssertions.assertSoftly(softly => {
+ softly.assertThat(mail1.getState).isEqualTo("transport")
+ softly.assertThat(mail2.getState).isEqualTo("error")
+ softly.assertThat(mail3.getState).isEqualTo("transport")
+ })
+ }
+
+ @Test
+ def shouldRateLimitSizeOfEmails(): Unit = {
+ val mailet = testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("duration", "20s")
+ .setProperty("size", "100K")
+ .build())
+
+ val mail1: Mail = FakeMail.builder()
+ .name("mail1")
+ .sender("sender@domain.tld")
+ .size(50 * 1024)
+ .state("transport")
+ .build()
+
+ val mail2: Mail = FakeMail.builder()
+ .name("mail2")
+ .sender("sender@domain.tld")
+ .size(51 * 1024)
+ .state("transport")
+ .build()
+
+ val mail3: Mail = FakeMail.builder()
+ .name("mail3")
+ .sender("sender@domain.tld")
+ .size(49 * 1024)
+ .state("transport")
+ .build()
+
+ mailet.service(mail1)
+ mailet.service(mail2)
+ mailet.service(mail3)
+
+ SoftAssertions.assertSoftly(softly => {
+ softly.assertThat(mail1.getState).isEqualTo("transport")
+ softly.assertThat(mail2.getState).isEqualTo("error")
+ softly.assertThat(mail3.getState).isEqualTo("transport")
+ })
+ }
+
+ @Test
+ def shouldRateLimitTotalSizeOfEmails(): Unit = {
+ val mailet = testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("duration", "20s")
+ .setProperty("totalSize", "1M")
+ .build())
+
+ val mail1: Mail = FakeMail.builder()
+ .name("mail1")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt2@linagora.com")
+ .size(50 * 1024)
+ .state("transport")
+ .build()
+
+ val mail2: Mail = FakeMail.builder()
+ .name("mail2")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt3@linagora.com", "rcpt4@linagora.com")
+ .size(311 * 1024)
+ .state("transport")
+ .build()
+
+ val mail3: Mail = FakeMail.builder()
+ .name("mail3")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt3@linagora.com")
+ .size(210 * 1024)
+ .state("transport")
+ .build()
+
+ mailet.service(mail1)
+ mailet.service(mail2)
+ mailet.service(mail3)
+
+ SoftAssertions.assertSoftly(softly => {
+ softly.assertThat(mail1.getState).isEqualTo("transport")
+ softly.assertThat(mail2.getState).isEqualTo("error")
+ softly.assertThat(mail3.getState).isEqualTo("transport")
+ })
+ }
+
+ @Test
+ def severalLimitsShouldBeApplied(): Unit = {
+ val mailet = testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("duration", "20s")
+ .setProperty("size", "100K")
+ .setProperty("recipients", "3")
+ .build())
+
+ val mail1: Mail = FakeMail.builder()
+ .name("mail1")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt2@linagora.com")
+ .size(50 * 1024)
+ .state("transport")
+ .build()
+
+ // Will exceed the size rate limit
+ val mail2: Mail = FakeMail.builder()
+ .name("mail2")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt3@linagora.com", "rcpt4@linagora.com")
+ .size(60 * 1024)
+ .state("transport")
+ .build()
+
+ // Will exceed the recipient rate limit
+ val mail3: Mail = FakeMail.builder()
+ .name("mail3")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com", "rcpt3@linagora.com")
+ .size(10 * 1024)
+ .state("transport")
+ .build()
+
+ // Will exceed the recipient rate limit
+ val mail4: Mail = FakeMail.builder()
+ .name("mail4")
+ .sender("sender@domain.tld")
+ .recipients("rcpt1@linagora.com")
+ .size(10 * 1024)
+ .state("transport")
+ .build()
+
+ mailet.service(mail1)
+ mailet.service(mail2)
+ mailet.service(mail3)
+ mailet.service(mail4)
+
+ SoftAssertions.assertSoftly(softly => {
+ softly.assertThat(mail1.getState).isEqualTo("transport")
+ softly.assertThat(mail2.getState).isEqualTo("error")
+ softly.assertThat(mail3.getState).isEqualTo("error")
+ softly.assertThat(mail4.getState).isEqualTo("transport")
+ })
+ }
+
+ @Nested
+ class Configuration {
+ @Test
+ def shouldFailWhenNoDuration(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "3")
+ .build()))
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWhenEmptyDuration(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "3")
+ .setProperty("duration", "")
+ .build()))
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWhenBadDuration(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "3")
+ .setProperty("duration", "bad")
+ .build()))
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWhenNegativeDuration(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "3")
+ .setProperty("duration", "-3s")
+ .build()))
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWhenZeroDuration(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "3")
+ .setProperty("duration", "0s")
+ .build()))
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWhenTooSmallDuration(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "3")
+ .setProperty("duration", "10ms")
+ .build()))
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def durationWithNoUnitShouldDefaultToSeconds(): Unit = {
+ assertThat(
+ testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "3")
+ .setProperty("duration", "10")
+ .build()).parseDuration().getSeconds)
+ .isEqualTo(10L)
+ }
+
+ @Test
+ def durationShouldSupportUnits(): Unit = {
+ assertThat(
+ testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "3")
+ .setProperty("duration", "1h")
+ .build()).parseDuration().getSeconds)
+ .isEqualTo(3600L)
+ }
+
+ @Test
+ def shouldFailWithEmptyRecipients(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithZeroRecipients(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "0")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithNegativeRecipients(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "-1")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithBadRecipients(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("recipients", "bad")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithEmptyCount(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("count", "")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithZeroCount(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("count", "0")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithNegativeCount(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("count", "-1")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithBadCount(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("count", "bad")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithEmptySize(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("size", "")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithZeroSize(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("size", "0")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithNegativeSize(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("size", "-1000")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithBadSize(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("size", "bad")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def sizeShouldSupportUnits(): Unit = {
+ assertThatCode(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("size", "1k")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .doesNotThrowAnyException()
+ }
+
+ @Test
+ def shouldFailWithEmptyTotalSize(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("totalSize", "")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithZeroTotalSize(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("totalSize", "0")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithNegativeTotalSize(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("totalSize", "-1000")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def shouldFailWithBadToatalSize(): Unit = {
+ assertThatThrownBy(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("totalSize", "bad")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .isInstanceOf(classOf[IllegalArgumentException])
+ }
+
+ @Test
+ def totalSizeShouldSupportUnits(): Unit = {
+ assertThatCode(() => testee(FakeMailetConfig.builder()
+ .mailetName("PerSenderRateLimit")
+ .setProperty("totalSize", "1k")
+ .setProperty("duration", "10s")
+ .build()).parseDuration().getSeconds)
+ .doesNotThrowAnyException()
+ }
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 04/05: JAMES-3683 Reduce size of the rate limiting plugin by excluding James dependencies
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 2e2a6942d3c6f28105814e4eadf96de924792786
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Wed Jan 19 20:39:42 2022 +0700
JAMES-3683 Reduce size of the rate limiting plugin by excluding James dependencies
57MB -> 203KB
---
server/mailet/rate-limiter/pom.xml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/server/mailet/rate-limiter/pom.xml b/server/mailet/rate-limiter/pom.xml
index 6c78008..cbec14e 100644
--- a/server/mailet/rate-limiter/pom.xml
+++ b/server/mailet/rate-limiter/pom.xml
@@ -37,10 +37,12 @@
<dependency>
<groupId>${james.groupId}</groupId>
<artifactId>apache-mailet-api</artifactId>
+ <scope>provided</scope>
</dependency>
<dependency>
<groupId>${james.groupId}</groupId>
<artifactId>apache-mailet-base</artifactId>
+ <scope>provided</scope>
</dependency>
<dependency>
<groupId>${james.groupId}</groupId>
@@ -50,6 +52,7 @@
<dependency>
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
+ <scope>provided</scope>
</dependency>
<dependency>
<groupId>es.moki.ratelimitj</groupId>
@@ -59,14 +62,17 @@
<dependency>
<groupId>eu.timepit</groupId>
<artifactId>refined_${scala.base}</artifactId>
+ <scope>provided</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-core</artifactId>
+ <scope>provided</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-scala-extensions_${scala.base}</artifactId>
+ <scope>provided</scope>
</dependency>
</dependencies>
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org