You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by "quantranhong1999 (via GitHub)" <gi...@apache.org> on 2023/05/05 04:03:08 UTC
[GitHub] [james-project] quantranhong1999 commented on a diff in pull request #1548: [PERF] Improved documentation + disable JMAP / ACLs
quantranhong1999 commented on code in PR #1548:
URL: https://github.com/apache/james-project/pull/1548#discussion_r1185678230
##########
server/apps/distributed-app/docs/modules/ROOT/pages/operate/performanceChecklist.adoc:
##########
@@ -0,0 +1,164 @@
+= Distributed James Server — Performance checklist
+:navtitle: Performance checklist
+
+This guide aims to help James operators refine their James configuration and set up to achieve better performance.
+
+== Database setup
+
+Cassandra, OpenSearch, RabbitMQ is a large topic in itself that we do not intend to cover here. Yet, here are some
+very basic recommendation that are always beneficial to keep in mind.
+
+We recommend:
+
+* Running Cassandra, OpenSearch on commodity hardware with attached SSD. SAN disks are known to cause performance
+issues for these technologies. HDD disks are to be banned for these performance related applications.
+* We recommend getting an Object Storage SaaS offering that suites your needs. Most generalist S3 offers will suite
+James needs.
+* We do provide a guide on xref:[Database benchmarks] that can help identify and fix issues.
+
+== James configuration
+
+=== Cassandra
+
+People tunning for performance would likely accept relaxing their consistency needs. James allows doing this.
+
+**LightWeight Transactions (LWT)** can be disabled where they are not essential. This can be done within
+xref:[cassandra.properties]:
+
+....
+mailbox.read.strong.consistency=false
+message.read.strong.consistency=false
+message.write.strong.consistency.unsafe=false
+mailrepository.strong.consistency=false
+....
+
+Also, James allows for **Read repairs** where consistency checks are piggy backed on reads randomly. This of course
+comes at a performance cost as it generates extre reads, thus minimizing read repair probability can help improving
+performance. This can be done within
+xref:[cassandra.properties]:
+
+....
+mailbox.read.repair.chance=0.00
+mailbox.counters.read.repair.chance.max=0.000
+mailbox.counters.read.repair.chance.one.hundred=0.000
+....
+
+One can also avoid some Cassandra requests by disabling ACLs (meaning users will only access to the mailboxes they own,
+all mailbox-sharing features will thus not be achievable). This can be done within
+xref:[cassandra.properties]:
+
+....
+acl.enabled=false
+....
+
+Important settings in the `` file includes:
Review Comment:
```suggestion
Important settings in the `cassandra-driver.conf` file includes:
```
##########
server/apps/distributed-app/docs/modules/ROOT/pages/operate/performanceChecklist.adoc:
##########
@@ -0,0 +1,164 @@
+= Distributed James Server — Performance checklist
+:navtitle: Performance checklist
+
+This guide aims to help James operators refine their James configuration and set up to achieve better performance.
+
+== Database setup
+
+Cassandra, OpenSearch, RabbitMQ is a large topic in itself that we do not intend to cover here. Yet, here are some
+very basic recommendation that are always beneficial to keep in mind.
+
+We recommend:
+
+* Running Cassandra, OpenSearch on commodity hardware with attached SSD. SAN disks are known to cause performance
+issues for these technologies. HDD disks are to be banned for these performance related applications.
+* We recommend getting an Object Storage SaaS offering that suites your needs. Most generalist S3 offers will suite
+James needs.
+* We do provide a guide on xref:[Database benchmarks] that can help identify and fix issues.
+
+== James configuration
+
+=== Cassandra
+
+People tunning for performance would likely accept relaxing their consistency needs. James allows doing this.
+
+**LightWeight Transactions (LWT)** can be disabled where they are not essential. This can be done within
+xref:[cassandra.properties]:
+
+....
+mailbox.read.strong.consistency=false
+message.read.strong.consistency=false
+message.write.strong.consistency.unsafe=false
+mailrepository.strong.consistency=false
+....
+
+Also, James allows for **Read repairs** where consistency checks are piggy backed on reads randomly. This of course
+comes at a performance cost as it generates extre reads, thus minimizing read repair probability can help improving
+performance. This can be done within
+xref:[cassandra.properties]:
+
+....
+mailbox.read.repair.chance=0.00
+mailbox.counters.read.repair.chance.max=0.000
+mailbox.counters.read.repair.chance.one.hundred=0.000
+....
+
+One can also avoid some Cassandra requests by disabling ACLs (meaning users will only access to the mailboxes they own,
+all mailbox-sharing features will thus not be achievable). This can be done within
+xref:[cassandra.properties]:
+
+....
+acl.enabled=false
+....
+
+Important settings in the `` file includes:
+
+* Throttling: if too low then the Cassandra cluster is under-utilized. If too high, request bursts can cause significant
+Cassandra overload.
+
+....
+ advanced.throttler {
+ class = org.apache.james.backends.cassandra.utils.LockLessConcurrencyLimitingRequestThrottler
+
+ max-queue-size = 10000
+
+ max-concurrent-requests = 192
+ }
+....
+
+=== Object storage
+
+We recommend the use of the blob store cache, which will be populated by email headers which shall be treated as metadata.
+
+`blob.properties`:
+
+....
+cache.enable=true
+cache.cassandra.ttl=1year
+cache.cassandra.timeout=100ms
+cache.sizeThresholdInBytes=16 KiB
+....
+
+=== RabbitMQ
+
+We recommend against the use of the CassandraMailQueueView, as browsing and advanced queue management features
+is unnecessary for Mail Delivery Agent and are not meaningful in the absence of delays.
+
+Similarly, we recommend turning off queue size metrics, which are expensive to compute.
+
+We also recommend against the use of publish confirms, which comes at a high performance price.
+
+In `rabbitmq.properties`:
+
+....
+cassandra.view.enabled=false
+
+mailqueue.size.metricsEnabled=false
+
+event.bus.publish.confirm.enabled=false
+mailqueue.publish.confirm.enabled=false
+....
+
+=== JMAP protocol
+
+If you are not using JMAP, disabling it will avoid you the cost of populating related projections and thus is recommended.
+Within `jmap.properties`:
+
+....
+enabled=false
+....
+
+We recommend turning on EmailQueryView as it enables resolution of mailbox listing against Cassandra, thus unlocking massive
+stability / performance gains. Within `jmap.properties`:
+
+....
+view.email.query.enabled=true
+....
+
+=== IMAP / SMTP
+
+We recommend against resolving client connection DNS names. This behaviour can be disabled via a system property within
+`jvm.properties`:
+
+....
+james.protocols.mdc.hostname=false
+....
+
+Concurrent IMAP request count is the critical setting. In `imapServer.xml`:
+
+....
+<concurrentRequests>200</concurrentRequests>
+<maxQueueSize>4096</maxQueueSize>
+....
+
+Other recommendation includes avoiding unecessary work upon IMAP IDLE, not starting dedicated BOSS threads:
+
+....
+<ignoreIDLEUponProcessing>false</ignoreIDLEUponProcessing>
+<bossWorkerCount>0</bossWorkerCount>
+....
+
+=== Other generic recommendations
+
+* Remove unneeded listeners / mailets
+* Reduce duplication of Matchers within mailetcontainer.xml
+* Limit usage of "DEBUG" loglevel. INFO should be more than decent in most cases.
+* While GC tunning is a science in itself, we had good results with G1GC and a low pause time:
Review Comment:
Not related to the PR but this reminds me to :p
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org