You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by GitBox <gi...@apache.org> on 2022/05/30 03:32:43 UTC
[GitHub] [james-project] chibenwa opened a new pull request, #1030: [PERF] Improve SMTP performance
chibenwa opened a new pull request, #1030:
URL: https://github.com/apache/james-project/pull/1030
This change set proposes a 38% performance improvement for the
following workload (reduced CPU usage):
```
smtp-source -A -C100 -r 1 -l 500000 -m 5000 -s 20 -d -c -f any@other.com -M other.com -t firstname14.surname14@upn.integration-open-paas.org localhost:25
```
Rationals:
- Submitted emails are read line by line and a couple of session
scoped values where retrieved from session attachment for each line:
mail size, header reached, etc. This causes not cheap HashMap reads
for each email line. This can easily be replaced by
getters/setters on the SMTP session to save the map read/writes.
- The data was accessed with the following transformation chain:
ByteBuf -> NIO byte buffer -> byte[]. But expensive reference
counting is triggered by Netty under the hood. Doing directly
ByteBuf -> byte[] prevents this.
- SMTP commands metric was incremented on each body line (!) which
also incurs a performance hit.
- Immutable handlers where retrieved on each request while we could
save them to fields to avoid re-computation.
- Avoid re-parsing charset.
- Use Deque::peekFirst instead of Iterables::getFirst when looking
up for line overrides (same than IMAP).
- Compute debug logs only if needed.
- Fasten slightly parsing SMTP parameters
This can be seen in the follwoing flame graph:
[smtp-cpu.zip](https://github.com/apache/james-project/files/8795220/smtp-cpu.zip)
## TODO
Flame graph after!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] Arsnael commented on pull request #1030: [PERF] Improve SMTP performance
Posted by GitBox <gi...@apache.org>.
Arsnael commented on PR #1030:
URL: https://github.com/apache/james-project/pull/1030#issuecomment-1141760499
https://ci-builds.apache.org/job/james/job/ApacheJames/job/PR-1030/5/testReport/?cloudbees-analytics-link=scm-reporting%2Ftests%2Ffailed
I think those failed tests are related to this PR, please check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] quantranhong1999 commented on a diff in pull request #1030: [PERF] Improve SMTP performance
Posted by GitBox <gi...@apache.org>.
quantranhong1999 commented on code in PR #1030:
URL: https://github.com/apache/james-project/pull/1030#discussion_r884622154
##########
protocols/smtp/src/main/java/org/apache/james/protocols/smtp/core/DataLineMessageHookHandler.java:
##########
@@ -58,16 +57,15 @@ public class DataLineMessageHookHandler implements DataLineFilter, ExtensibleHan
private List<?> rHooks;
@Override
- public Response onLine(SMTPSession session, ByteBuffer line, LineHandler<SMTPSession> next) {
+ public Response onLine(SMTPSession session, byte[] line, LineHandler<SMTPSession> next) {
MailEnvelope env = session.getAttachment(DataCmdHandler.MAILENV, ProtocolSession.State.Transaction)
.orElseThrow(() -> new RuntimeException("'" + DataCmdHandler.MAILENV.asString() + "' has not been filled."));
OutputStream out = getMessageOutputStream(env);
try {
// 46 is "."
- // Stream terminated
- int c = line.get();
- if (line.remaining() == 2 && c == 46) {
+ // Stream terminated
+ if (line.length == 3 && line[0] == 46) {
Review Comment:
```suggestion
if (line.length == 2 && line[0] == 46) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #1030: [PERF] Improve SMTP performance
Posted by GitBox <gi...@apache.org>.
chibenwa commented on PR #1030:
URL: https://github.com/apache/james-project/pull/1030#issuecomment-1142117754
Broken tests for LmtpServerTest
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #1030: [PERF] Improve SMTP performance
Posted by GitBox <gi...@apache.org>.
chibenwa commented on PR #1030:
URL: https://github.com/apache/james-project/pull/1030#issuecomment-1143219721
```
Before
5000 messages of 500K -> 1min57
5000 messages of 50K -> 1min25
5000 messages of 5K -> 1min24
AFTER
5000 messages of 500K -> 1min27
5000 messages of 50K -> 1min21
5000 messages of 5K -> 1min20
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa merged pull request #1030: [PERF] Improve SMTP performance
Posted by GitBox <gi...@apache.org>.
chibenwa merged PR #1030:
URL: https://github.com/apache/james-project/pull/1030
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org