You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by GitBox <gi...@apache.org> on 2020/12/31 03:37:37 UTC
[GitHub] [james-project] chibenwa opened a new pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
chibenwa opened a new pull request #285:
URL: https://github.com/apache/james-project/pull/285
…shold
This governs the threshold MimeMessageInputStreamSource relies
on for storing MimeMessage content on disk.
Below, data is stored in memory. Above data is stored on disk.
Lower values will lead to longer processing time but will
minimize heap memory usage. Modern SSD hardware
should however support a high throughput. Higher values
will lead to faster single mail processing at the cost
of higher heap usage.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server, on my laptop. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold | | |
| 100KB threshold | | 162s |
| 500 KB threshold | | 159s |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server, on my laptop. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold (aka file) | 214s | 193s |
| 100KB threshold (aka mixed) | 173s | 181s |
| 500 KB threshold (memory)| 171s | 159s |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-755293554
I see, your comment makes much more sense in the context of https://github.com/apache/james-project/pull/282 and possibly https://github.com/apache/james-project/pull/286
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on a change in pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on a change in pull request #285:
URL: https://github.com/apache/james-project/pull/285#discussion_r550433315
##########
File path: docs/modules/servers/pages/distributed/configure/index.adoc
##########
@@ -59,4 +59,27 @@ By omitting these files, no extra behaviour is added.
** xref:distributed/configure/spam.adoc[This page] documents Anti-Spam setup with SpamAssassin.
** xref:distributed/configure/remote-delivery-error-handling.adoc[This page] proposes a simple strategy for RemoteDelivery error handling.
** xref:distributed/configure/collecting-contacts.adoc[This page] documents contact collection
-** xref:distributed/configure/collecting-events.adoc[This page] documents event collection
\ No newline at end of file
+** xref:distributed/configure/collecting-events.adoc[This page] documents event collection
+
+== System properties
+
+Some tuning can be done via system properties. This includes:
+
+.System properties
+|===
+| Property name | explanation
+
+| james.message.memory.threshold
+| (Optional). String (size, integer + size units, example: `12 KIB`, supported units are bytes KIB MIB GIB TIB). Defaults to 100KIB.
+This governs the threshold MimeMessageInputStreamSource relies on for storing MimeMessage content on disk.
+Below, data is stored in memory. Above data is stored on disk.
+Lower values will lead to longer processing time but will minimize heap memory usage. Modern SSD hardware
+should however support a high throughput. Higher values will lead to faster single mail processing at the cost
+of higher heap usage.
+
+
+| james.message.usememorycopy
+|Optional. Boolean. Defaults to false. Recommended value is true.
Review comment:
that is a typo. Of course MUST be false
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server, on my laptop. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold (aka file) | | 193s |
| 100KB threshold (aka mixed) | 173s | 181s |
| 500 KB threshold (memory)| 171ms | 159s |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-811842757
To be honnest we fail to reach a consensus, there's no real alternative proposals formulated to the java properties so far.
As I personally do not really care, I will close that work.
Please, kind contributors, if you are interested in customizing these constants feel free to take over this work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] rouazana commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
rouazana commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-755283579
Maybe I just did not comment on the right PR. If I got it correctly we are changing the default behavior to be more reliable, but it could have potential big performance impact. I would like to be reassured. But that's not specific to this PR, I will add a message on the mailing list thread.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-755258458
> can you detail your concerns ? as far as I can see the default has not changed ...
in theory the computation of the value for threshold could have an impact but it should get inlined pretty quickly since in a production setup the method call is always going to return the same value
IMO @rouazana concern as far as I understand was more about a Linagora adoption of a non-default value than on the impact of the system property call.
Here the scope is "we want to onfigure this." not "we want to change the default value".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold | | |
| 100KB threshold | | 162s |
| 500 KB threshold | | 159s |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #285:
URL: https://github.com/apache/james-project/pull/285#discussion_r550432532
##########
File path: docs/modules/servers/pages/distributed/configure/index.adoc
##########
@@ -59,4 +59,27 @@ By omitting these files, no extra behaviour is added.
** xref:distributed/configure/spam.adoc[This page] documents Anti-Spam setup with SpamAssassin.
** xref:distributed/configure/remote-delivery-error-handling.adoc[This page] proposes a simple strategy for RemoteDelivery error handling.
** xref:distributed/configure/collecting-contacts.adoc[This page] documents contact collection
-** xref:distributed/configure/collecting-events.adoc[This page] documents event collection
\ No newline at end of file
+** xref:distributed/configure/collecting-events.adoc[This page] documents event collection
+
+== System properties
+
+Some tuning can be done via system properties. This includes:
+
+.System properties
+|===
+| Property name | explanation
+
+| james.message.memory.threshold
+| (Optional). String (size, integer + size units, example: `12 KIB`, supported units are bytes KIB MIB GIB TIB). Defaults to 100KIB.
+This governs the threshold MimeMessageInputStreamSource relies on for storing MimeMessage content on disk.
+Below, data is stored in memory. Above data is stored on disk.
+Lower values will lead to longer processing time but will minimize heap memory usage. Modern SSD hardware
+should however support a high throughput. Higher values will lead to faster single mail processing at the cost
+of higher heap usage.
+
+
+| james.message.usememorycopy
+|Optional. Boolean. Defaults to false. Recommended value is true.
Review comment:
> Recommended value is true
I think the recommended value should be false as is the default, copy via MimeMessageInputStreamSource which leverages OS disk write caches through DeferredFileOutputStream thus letting the OS decide to use memory or disk (trading speed for memory) base on system load and memory pressure.
##########
File path: docs/modules/servers/pages/distributed/configure/index.adoc
##########
@@ -59,4 +59,27 @@ By omitting these files, no extra behaviour is added.
** xref:distributed/configure/spam.adoc[This page] documents Anti-Spam setup with SpamAssassin.
** xref:distributed/configure/remote-delivery-error-handling.adoc[This page] proposes a simple strategy for RemoteDelivery error handling.
** xref:distributed/configure/collecting-contacts.adoc[This page] documents contact collection
-** xref:distributed/configure/collecting-events.adoc[This page] documents event collection
\ No newline at end of file
+** xref:distributed/configure/collecting-events.adoc[This page] documents event collection
+
+== System properties
+
+Some tuning can be done via system properties. This includes:
+
+.System properties
+|===
+| Property name | explanation
+
+| james.message.memory.threshold
+| (Optional). String (size, integer + size units, example: `12 KIB`, supported units are bytes KIB MIB GIB TIB). Defaults to 100KIB.
+This governs the threshold MimeMessageInputStreamSource relies on for storing MimeMessage content on disk.
+Below, data is stored in memory. Above data is stored on disk.
+Lower values will lead to longer processing time but will minimize heap memory usage. Modern SSD hardware
+should however support a high throughput. Higher values will lead to faster single mail processing at the cost
+of higher heap usage.
+
+
+| james.message.usememorycopy
+|Optional. Boolean. Defaults to false. Recommended value is true.
Review comment:
> Recommended value is true
I think the recommended value should be false as is the default, copy via MimeMessageInputStreamSource which leverages OS disk write caches through DeferredFileOutputStream thus letting the OS decide to use memory or disk (trading speed for memory) base on system load and memory pressure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server. Here are the results:
|----------------|----------------|--------------------|
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold | | |
| 100KB threshold | | |
| 500 KB threshold | | |
|----------------|----------------|--------------------|
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-752895549
I'll see if I find some tine to do some bench ;-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-755255428
>Could we have some real (aka Gatling) bench please? I'm feeling very concerned by the potential impact on production.
can you detail your concerns ? as far as I can see the default has not changed ...
in theory the computation of the value for threshold could have an impact but it should get inlined pretty quickly since in a production setup the method call is always going to return the same value
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa closed pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa closed pull request #285:
URL: https://github.com/apache/james-project/pull/285
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-811842757
To be honest we fail to reach a consensus, there's no real alternative proposals formulated to the java properties so far.
As I personally do not really care, I will close that work.
Please, kind contributors, if you are interested in customizing these constants feel free to take over this work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-876856663
While having a round at open tickets, I find back this one.
We encountered a similar problem in https://github.com/apache/james-project/pull/450 and addressed it via an environment variable.
I wonder if we could not do the same here...
Cc @jeantil @mbaechler
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server, on my laptop. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold (aka file) | | 193s |
| 100KB threshold (aka mixed) | | 181s |
| 500 KB threshold (memory)| | 159s |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-755257031
@mbaechler sure no problem, but which conf file would you recomend? `core.properties` ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold | | |
| 100KB threshold | | 162s |
| 500 KB threshold | | |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on a change in pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on a change in pull request #285:
URL: https://github.com/apache/james-project/pull/285#discussion_r550433315
##########
File path: docs/modules/servers/pages/distributed/configure/index.adoc
##########
@@ -59,4 +59,27 @@ By omitting these files, no extra behaviour is added.
** xref:distributed/configure/spam.adoc[This page] documents Anti-Spam setup with SpamAssassin.
** xref:distributed/configure/remote-delivery-error-handling.adoc[This page] proposes a simple strategy for RemoteDelivery error handling.
** xref:distributed/configure/collecting-contacts.adoc[This page] documents contact collection
-** xref:distributed/configure/collecting-events.adoc[This page] documents event collection
\ No newline at end of file
+** xref:distributed/configure/collecting-events.adoc[This page] documents event collection
+
+== System properties
+
+Some tuning can be done via system properties. This includes:
+
+.System properties
+|===
+| Property name | explanation
+
+| james.message.memory.threshold
+| (Optional). String (size, integer + size units, example: `12 KIB`, supported units are bytes KIB MIB GIB TIB). Defaults to 100KIB.
+This governs the threshold MimeMessageInputStreamSource relies on for storing MimeMessage content on disk.
+Below, data is stored in memory. Above data is stored on disk.
+Lower values will lead to longer processing time but will minimize heap memory usage. Modern SSD hardware
+should however support a high throughput. Higher values will lead to faster single mail processing at the cost
+of higher heap usage.
+
+
+| james.message.usememorycopy
+|Optional. Boolean. Defaults to false. Recommended value is true.
Review comment:
that a typo. Of course MUST be false
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-778892607
I will not invest time to introduce yet-another configuration file - we already have too much of them.
@mbaechler I see no viable alternative proposed hence I will follow up with this approach.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-755257415
> Could we have some real (aka Gatling) bench please? I'm feeling very concerned by the potential impact on production.
I guess people ending up relying on non default properties would run suh benchs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold | | |
| 100KB threshold | | |
| 500 KB threshold | | |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-778892607
I will not invest time to introduce yet-another configuration file - we already have too much of them.
@mbaechler I see no viable alternative proposed hence I will follow up with this approach.
Anyway, not running James as a MX I do personally not need strongly this customisation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server, on my laptop. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold (aka file) | | 193s |
| 100KB threshold (aka mixed) | | 162s |
| 500 KB threshold (memory)| | 159s |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server, on my laptop. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold (aka file) | | 193s |
| 100KB threshold (aka mixed) | 173s | 181s |
| 500 KB threshold (memory)| | 159s |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa edited a comment on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa edited a comment on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-753440706
I had been running a quick benchmark, sending 2000 mails of 100KB over 10 threads, on top of the distributed server, on my laptop. Here are the results:
| Code version | master | #286 |
|----------------|----------------|--------------------|
| 1 byte threshold (aka file) | 214s | 193s |
| 100KB threshold (aka mixed) | 180s | 181s |
| 500 KB threshold (memory)| 171s | 159s |
I did not manage to evaluate heap impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] rouazana commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
rouazana commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-755246477
Could we have some real (aka Gatling) bench please? I'm feeling very concerned by the potential impact on production.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #285:
URL: https://github.com/apache/james-project/pull/285#issuecomment-780238713
test this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #285: JAMES-3487 Java property to control MimeMessageInputStreamSource thre…
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #285:
URL: https://github.com/apache/james-project/pull/285#discussion_r550432766
##########
File path: src/site/xdoc/server/config-system.xml
##########
@@ -151,6 +151,27 @@
</subsection>
+ <subsection name="System properties">
+
+ <p>Some tuning can be done via system properties. This includes:</p>
+
+ <dl>
+ <dt><strong>james.message.memory.threshold</strong></dt>
+ <dd>(Optional). String (size, integer + size units, example: `12 KIB`, supported units are bytes KIB MIB GIB TIB). Defaults to 100KIB. <br/>
+ This governs the threshold MimeMessageInputStreamSource relies on for storing MimeMessage content on disk. <br/>
+ Below, data is stored in memory. Above data is stored on disk.
+ Lower values will lead to longer processing time but will minimize heap memory usage. Modern SSD hardware
+ should however support a high throughput. Higher values will lead to faster single mail processing at the cost
+ of higher heap usage.
+ </dd>
+
+ <dt><strong>james.message.usememorycopy</strong></dt>
+ <dd>Optional. Boolean. Defaults to false. Recommended value is true. <br/>
Review comment:
same as above on recommended value
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org