You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by GitBox <gi...@apache.org> on 2021/05/16 04:26:03 UTC
[GitHub] [james-project] chibenwa opened a new pull request #434: JAMES-3107 Desactivate metrics p99 log
chibenwa opened a new pull request #434:
URL: https://github.com/apache/james-project/pull/434
Using metrics to capture p99 lead to over-snapshoting, and incurs a 33% throughtput penalty on JMAP draft.
Other SLOW logging implementations do rely on manual time
measurements (eg Datastax Cassandra driver).
As such, due to the low benefits and high costs I propose
to deprecate the logP99 API and migrate away from it.
Tools like Glowroot are able to capture slow traces at a
lighter cost and should rather be used.
## Performance
I did run 2 benchmarks to evaluate the changes in terms of performance:
- a. Run 1000 users doing a representative JMAP polling request with a pause of 2-4 seconds in between. This represents the regular usage.
-b. Run 2000 users doing a representative JMAP polling request with a pause of 2-4 seconds in between. This adapts to system maximum throughtput (as high latencies would add up to the delays and acts as a feedback loop).
For each of these scenari we include:
- Gatling overview
- Glowroot average, that includes time spent executing Cassanra queries
- Glowroot percentiles
- Glowroot averages (1 min)
- And finally Glowroot Cassandra queries breakdown
Testing infrastructure:
- 3 James, 3 CPU, 6GB memory
- 3 Cassandra (SSD, 8 CPU, 32GB ram) - OVH B2-30)
- OVH S3 object store
The instances are pre-provisionned with representative email corpus. Each account have a representative mailbox distribution 50 -> 1000 mailboxes (hence Mailbox/get dispersion)
### Before
#### a. LOG p99 and Regular scenario
#### b. LOG p99 and maximum throughtput
### After
#### a. LOG p99 and Regular scenario
#### b. LOG p99 and maximum throughtput
### Conclusion
- Dramatic improvments of all KPIs at all levels of load. We can reach workloads with +50% higher
- Surprisingly enough, Cassandra queries executed faster. I think because of CPU shortage Cassandra query execution lead to delays.
Side note: I tried many variations of the metrics, and oddly enough the final one behave even better than a "logging" metric implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #434: JAMES-3107 Desactivate metrics p99 log
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #434:
URL: https://github.com/apache/james-project/pull/434#issuecomment-842856119
Rebased, I added one more commit which should be non-controversial...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] Arsnael commented on pull request #434: JAMES-3107 Desactivate metrics p99 log
Posted by GitBox <gi...@apache.org>.
Arsnael commented on pull request #434:
URL: https://github.com/apache/james-project/pull/434#issuecomment-842070427
Rebase please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa merged pull request #434: JAMES-3107 Desactivate metrics p99 log
Posted by GitBox <gi...@apache.org>.
chibenwa merged pull request #434:
URL: https://github.com/apache/james-project/pull/434
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] Arsnael commented on pull request #434: JAMES-3107 Desactivate metrics p99 log
Posted by GitBox <gi...@apache.org>.
Arsnael commented on pull request #434:
URL: https://github.com/apache/james-project/pull/434#issuecomment-842070427
Rebase please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org